Skip to content

Commit 203595a

Browse files
tothmanotothmano
authored
Update information on regions (#268)
1 parent e10e718 commit 203595a

File tree

3 files changed

+50
-261
lines changed

3 files changed

+50
-261
lines changed

‎reference/regions.mdx

+12-10
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,20 @@ The examples in this documentation use the US domain. If your organization uses
1111

1212
## Check your region
1313

14-
To check which region your organization uses, open the Axiom web app and check the URL in the browser:
15-
- If the URL starts with `https://app.axiom.co/`, you use the default US region.
16-
- If the URL starts with `https://app.eu.axiom.co/`, you use the EU region.
14+
To check which region your organization uses, match the base domain in the Axiom web app with the table below:
1715

18-
## Axiom app
19-
20-
If your organization uses the US region, the base domain is `https://app.axiom.co/`.
21-
22-
If your organization uses the EU region, the base domain is `https://app.eu.axiom.co/`. This is different from the default US region `https://app.axiom.co/` you see everywhere in the documentation.
16+
| Region | Base domain in web app |
17+
| ---------- | ------------------------------ |
18+
| US | `https://app.axiom.co` |
19+
| EU | `https://app.eu.axiom.co` |
2320

2421
## Axiom API reference
2522

26-
All examples in the [Axiom API reference](/restapi/introduction) use the default US base domain `https://api.axiom.co`. For example, if your organization uses the US region, send data to Axiom with the URL `https://api.axiom.co/v1/datasets/{id}/ingest`.
23+
All examples in the documentation use the default US base domain `https://api.axiom.co`.
24+
25+
If your organization uses the EU region, change the base domain in the examples to `https://api.eu.axiom.co`.
2726

28-
If your organization uses the EU region, change the base domain in the examples to `https://api.eu.axiom.co`. For example, if your organization uses the EU region, send data to Axiom with the URL `https://api.eu.axiom.co/v1/datasets/{id}/ingest`.
27+
| Region | Base domain of API endpoints | Example |
28+
| ---------- | ------------------------------------ | ------------------------------------------------- |
29+
| US | `https://api.axiom.co` | `https://api.axiom.co/v1/datasets/{id}/ingest` |
30+
| EU | `https://api.eu.axiom.co` | `https://api.eu.axiom.co/v1/datasets/{id}/ingest` |

‎send-data/fluent-bit.mdx

+32-251
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: 'Send data from Fluent Bit to Axiom'
3-
description: 'This step-by-step guide will help you collect any data like metrics and logs from different sources, enrich them with filters, and send them to Axiom.'
3+
description: 'This page explains how to send data from Fluent Bit to Axiom.'
44
overview: 'Lightweight log processor and forwarder for various data sources'
55
sidebarTitle: Fluent Bit
66
tags:
@@ -11,253 +11,34 @@ popularityOrder: 11
1111
---
1212

1313
import ReplaceDatasetToken from "/snippets/replace-dataset-token.mdx"
14-
15-
Fluent Bit is an open-source Log Processor and Forwarder that allows you to collect any data like metrics and logs from different sources, enrich them with filters, and send them to multiple destinations like Axiom.
16-
17-
## Installation
18-
19-
Visit the [Fluent Bit download page](https://docs.fluentbit.io/manual/installation/getting-started-with-fluent-bit) to install Fluent Bit on your system.
20-
21-
You'd need to specify the org-id header if you are using personal token, it’s best to use an API token to avoid the need to specify the org-id header.
22-
23-
Learn more about [API and personal token](/reference/tokens)
24-
25-
## Configuration
26-
27-
Fluent Bit configuration file supports four types of sections:
28-
29-
- Service: Defines global properties of your service using different keys available for a specific version.
30-
- Input: Defines the input plugin and base configuration of your file.
31-
- Filter: Defines the input plugin and configure the pattern tags for your configuration.
32-
- Output: Specify a destination that certain records should follow after a Tag match.
33-
34-
All sections are configured in your `.conf` file.
35-
36-
## Example
37-
38-
The example below shows fluent Bit configuration that sends data to Axiom:
39-
40-
```ini
41-
[SERVICE]
42-
Flush 5
43-
Daemon off
44-
Log_Level debug
45-
46-
[INPUT]
47-
Name cpu
48-
Tag cpu
49-
50-
[OUTPUT]
51-
Name http
52-
Match *
53-
Host api.axiom.co
54-
Port 443
55-
URI /v1/datasets/DATASET_NAME/ingest
56-
# Authorization Bearer should be an API token
57-
Header Authorization Bearer API_TOKEN
58-
compress gzip
59-
format json
60-
json_date_key _time
61-
json_date_format iso8601
62-
tls On
63-
```
64-
65-
## Fluent Bit filters
66-
67-
Fluent Bit provides several filter plugins that can be used to modify the logs. These filters can be added to the configuration file in the `[FILTER]` section.
68-
69-
Here’s how you can do it:
70-
71-
## AWS ECS filter
72-
73-
For AWS ECS, you can use the `grep` filter which enriches logs with Amazon ECS metadata:
74-
75-
```ini
76-
[SERVICE]
77-
Flush 5
78-
Daemon off
79-
Log_Level debug
80-
81-
[INPUT]
82-
Name cpu
83-
Tag cpu
84-
85-
[FILTER]
86-
Name grep
87-
Match *
88-
Regex ecs_task_arn .*app1.*
89-
90-
[OUTPUT]
91-
Name http
92-
Match *
93-
Host api.axiom.co
94-
Port 443
95-
URI /v1/datasets/DATASET_NAME/ingest
96-
# Authorization Bearer should be an API token
97-
Header Authorization Bearer API_TOKEN
98-
compress gzip
99-
format json
100-
json_date_key _time
101-
json_date_format iso8601
102-
tls On
103-
```
104-
105-
<ReplaceDatasetToken />
106-
107-
## Kubernetes Filter
108-
109-
The `kubernetes` filter enriches logs with Kubernetes metadata:
110-
111-
```ini
112-
[SERVICE]
113-
Flush 5
114-
Daemon off
115-
Log_Level debug
116-
117-
[INPUT]
118-
Name cpu
119-
Tag cpu
120-
121-
[FILTER]
122-
Name kubernetes
123-
Match *
124-
Kube_URL https://kubernetes.default.svc:443
125-
Merge_Log On
126-
K8S-Logging.Parser On
127-
K8S-Logging.Exclude On
128-
129-
[OUTPUT]
130-
Name http
131-
Match *
132-
Host api.axiom.co
133-
Port 443
134-
URI /v1/datasets/DATASET_NAME/ingest
135-
# Authorization Bearer should be an API token
136-
Header Authorization Bearer API_TOKEN
137-
compress gzip
138-
format json
139-
json_date_key _time
140-
json_date_format iso8601
141-
tls On
142-
```
143-
144-
<ReplaceDatasetToken />
145-
146-
## WASM Filter
147-
148-
Fluent Bit allows the usage of WebAssembly (WASM) based filters.
149-
150-
```ini
151-
[SERVICE]
152-
Flush 5
153-
Daemon off
154-
Log_Level debug
155-
156-
[INPUT]
157-
Name cpu
158-
Tag cpu
159-
160-
[FILTER]
161-
Name wasm
162-
Match *
163-
Path /path/to/wasm/filter.wasm
164-
public_token xxxxxxxxxxx
165-
166-
[OUTPUT]
167-
Name http
168-
Match *
169-
Host api.axiom.co
170-
Port 443
171-
URI /v1/datasets/DATASET_NAME/ingest
172-
# Authorization Bearer should be an API token
173-
Header Authorization Bearer API_TOKEN
174-
compress gzip
175-
format json
176-
json_date_key _time
177-
json_date_format iso8601
178-
tls On
179-
```
180-
181-
<ReplaceDatasetToken />
182-
183-
## Send logs from Docker Compose with Fluent Bit
184-
185-
This section outlines how to configure Fluent Bit with Docker Compose to forward logs to Axiom. It includes setting up `fluent-bit.conf` for log processing and `docker-compose.yaml` for deploying Fluent Bit as a container. The setup captures logs from various system metrics, logs, and forwards them to Axiom.
186-
187-
### Create Fluent Bit configuration file (fluent-bit.conf)
188-
189-
```ini
190-
[SERVICE]
191-
Flush 1
192-
Daemon off
193-
Log_Level debug
194-
195-
[INPUT]
196-
Name cpu
197-
Tag system.cpu
198-
Interval_Sec 5
199-
200-
[INPUT]
201-
Name mem
202-
Tag system.mem
203-
Interval_Sec 5
204-
205-
[INPUT]
206-
Name forward
207-
Listen 0.0.0.0
208-
port 24224
209-
210-
[INPUT]
211-
Name netif
212-
Tag netif
213-
Interval_Sec 1
214-
Interval_NSec 0
215-
Interface eth0
216-
217-
[INPUT]
218-
Name disk
219-
Tag disk
220-
Interval_Sec 1
221-
Interval_NSec 0
222-
223-
[FILTER]
224-
Name record_modifier
225-
Match *
226-
Record hostname ${HOSTNAME}
227-
228-
[OUTPUT]
229-
Name http
230-
Match *
231-
Host api.axiom.co
232-
Port 443
233-
URI /v1/datasets/DATASET_NAME/ingest
234-
Header Authorization Bearer API_TOKEN
235-
Compress gzip
236-
Format json
237-
JSON_Date_Key _time
238-
JSON_Date_Format iso8601
239-
TLS On
240-
```
241-
242-
<ReplaceDatasetToken />
243-
244-
### Create Docker Compose file (docker-compose.yaml)
245-
246-
Ensure the `volumes` section correctly maps the `fluent-bit.conf` file to `/fluent-bit/etc/fluent-bit.conf` inside the container with read-only access.
247-
248-
```yaml
249-
version: '3'
250-
251-
services:
252-
fluentbit:
253-
image: fluent/fluent-bit:latest
254-
container_name: fluent-bit
255-
user: root # Required for accessing host log files
256-
volumes:
257-
- ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
258-
- /var/lib/docker/containers:/opt/docker-container-logs:ro
259-
environment:
260-
- AXIOM_HOSTNAME=axiom
261-
```
262-
263-
To start the Fluent Bit container using the Docker Compose configuration you've set up, execute the `docker-compose up -d` command.
14+
import Prerequisites from "/snippets/standard-prerequisites.mdx"
15+
16+
Fluent Bit is an open-source log processor and forwarder that allows you to collect any data like metrics and logs from different sources, enrich them with filters, and send them to multiple destinations like Axiom.
17+
18+
<Prerequisites />
19+
{/* list separator */}
20+
- [Install Fluent Bit](https://docs.fluentbit.io/manual/installation/getting-started-with-fluent-bit).
21+
22+
## Configure Fluent Bit
23+
24+
1. Set up the Fluent Bit configuration file based on the [Fluent Bit documentation](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/configuration-file).
25+
1. In the Fluent Bit configuration file, use the HTTP output plugin with the following configuration. For more information on the plugin, see the [Fluent Bit documentation](https://docs.fluentbit.io/manual/pipeline/outputs/http).
26+
27+
```ini
28+
[OUTPUT]
29+
Name http
30+
Match *
31+
Host AXIOM_URL
32+
Port 443
33+
URI /v1/datasets/DATASET_NAME/ingest
34+
Header Authorization Bearer API_TOKEN
35+
Compress gzip
36+
Format json
37+
JSON_Date_Key _time
38+
JSON_Date_Format iso8601
39+
TLS On
40+
```
41+
42+
- Replace `AXIOM_URL` with `api.axiom.co` if your organization uses the US region, and with `api.eu.axiom.co` if your organization uses the EU region. For more information, see [Regions](/reference/regions).
43+
{/* list separator */}
44+
<ReplaceDatasetToken />

‎send-data/ingest.mdx

+6
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,12 @@ You can create an endpoint for the following services and send the logs directly
287287
<img src="/doc-assets/shots/endpoints-356.png" />
288288
</Frame>
289289

290+
## Regions
291+
292+
All examples in the documentation use the base domain `https://api.axiom.co`, which is the default for the US region. If your organization uses the EU region, change the base domain in the examples to `https://api.eu.axiom.co`.
293+
294+
For more information on regions, see [Regions](/reference/regions).
295+
290296
## Limits and requirements
291297

292298
Axiom applies certain limits and requirements on the ingested data to guarantee good service across the platform. Some of these limits depend on your pricing plan, and some of them are applied system-wide. For more information, see [Limits and requirements](/reference/field-restrictions).

0 commit comments

Comments
 (0)