Elastic Agent Configuration

To get started first follow the steps below:

• Install Elastic Agent
• Root access
• Verify the required port is open

Older versions can be found here 7

The configuration file below is pre-configured to send data to your Logit.io Stack via Logstash.

Copy the configuration file below and overwrite the contents of elastic-agent.yml. For version 7.17+ and above, use the configuration below:

###################### Logit.io Elastic Agent Configuration ########################
# ============================== Elastic Agent inputs ==============================
inputs:
  - type: system/metrics
    id: unique-system-metrics-input
    data_stream.namespace: default
    use_output: default
    streams:
      - metricsets:
        - cpu
        data_stream.dataset: system.cpu
      - metricsets:
        - memory
        data_stream.dataset: system.memory
      - metricsets:
        - network
        data_stream.dataset: system.network
      - metricsets:
        - filesystem
        data_stream.dataset: system.filesystem

  #   Collecting log files
  #   - type: filestream
  #     id: your-input-id
  #     streams:
  #       # Must be unique
  #       - id: your-filestream-stream-id
  #         data_stream:
  #           dataset: generic
  #         paths:
  #           - /var/log/*.log             

# ================================== Outputs ===================================
# ------------------------------ Logstash Output -------------------------------
outputs:
  default:
    type: logstash
    hosts: ["your-logstash-host:your-ssl-port"]
    loadbalance: true
    ssl.enabled: true

# ================================== Logging Settings ===================================
# Send all logging output to stderr, default is false
agent.logging.to_stderr: true

Validate your YAML

It’s a good idea to run the configuration file through a YAML validator to rule out indentation errors, clean up extra characters, and check if your YAML file is valid. Yamllint.com is a great choice.

1. User and Process Auditing:

• Regularly examine and assess the actions performed by users and processes on your systems.

2. Operating System and Service Metrics:

• Gather performance metrics from the operating systems and services running on your servers to track and optimize their functioning.

3. Log Data Forwarding and Centralization:

• Channel and consolidate log data from various sources to a central location for easier analysis and management.

4. Service Monitoring:

• Keep track of the status of your services to ensure they are functioning as intended.

5. Network Traffic Monitoring:

• Monitor the flow of network traffic between servers in your network to identify and address any irregularities or potential security threats.

Read more about Elastic Agent inputs for the list of what’s available

DEB/RPM

sudo elastic-agent -e -c /etc/elastic-agent/elastic-agent.yml

macOS

sudo ./elastic-agent -e -c elastic-agent.yml --strict.perms=false

You’ll be running elastic-agent as root, so you need to change ownership of the configuration file and any configurations enabled in the modules.d directory, or run elastic-agent with --strict.perms=false as shown above. Read more about how to change ownership.

Windows

cd <EXTRACTED_ARCHIVE>
.\elastic-agent.exe -e -c elastic-agent.yml

Linux

sudo ./elastic-agent -e -c elastic-agent.yml --strict.perms=false

You’ll be running elastic-agent as root, so you need to change ownership of the configuration file and any configurations enabled in the modules.d directory, or run elastic-agent with --strict.perms=false as shown above. Read more about how to change ownership.

If the yml file is invalid, elastic-agent will print an `error loading config file` error message with details on how to correct the problem. If you have issues starting elastic-agent see "How To Diagnose No Data In Stack" below to troubleshoot.

Start or Restart elastic-agent to apply the configuration changes.

Data should now have been sent to your Stack.

View my data

If you don't see logs take a look at How to diagnose no data in Stack below for how to diagnose common issues.

If you don't see data appearing in your Stack after following the steps, visit the Help Centre guide for steps to diagnose no data appearing in your Stack or Chat to support now.

Elastic Agent is a versatile and reliable tool designed for efficient log and metric ingestion into OpenSearch and other destinations within the OpenSearch Stack. It represents a fusion of capabilities from various Elastic Stack components, offering a unified solution for data collection.

With Elastic Agent, you can generate, parse, and forward logs and metrics seamlessly, ensuring their proper indexing within Elasticsearch. It excels at handling different data types and provides a streamlined approach to data ingestion.

It's important to note that Elastic Agent complements Logstash, making them an effective combination for complex data pipelines. While initially compatible with Elasticsearch, Elastic Agent's capabilities have expanded to include integration with technologies like Redis and Kafka.

Proper configuration of Elastic Agent is essential to avoid complex logging issues, such as excessively large registry files or errors related to deleted or renamed log files. A well-configured agent ensures smooth data collection and indexing.

If you need any further assistance with migrating your log data to ELK we're here to help you get started. Feel free to get in contact with our support team by sending us a message via live chat & we'll be happy to assist.