Skip to content

Commit 16292bf

Browse files
Merge pull request #2136 from segmentio/DOC-359-IF
Data Lakes Reports Documentation [DOC-359]
2 parents e3a70f5 + 074722a commit 16292bf

File tree

2 files changed

+53
-0
lines changed

2 files changed

+53
-0
lines changed

src/_data/sidenav/main.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,8 @@ sections:
188188
title: Set Up Data Lakes
189189
- path: /connections/storage/data-lakes/sync-reports
190190
title: Sync Reports and Error Reporting
191+
- path: /connections/storage/data-lakes/sync-history
192+
title: Data Lakes Sync History and Health
191193
- path: /connections/storage/data-lakes/comparison
192194
title: Data Lakes vs. Warehouses
193195
- section_title: Data Warehouses
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
title: Data Lakes Sync History and Health
3+
---
4+
{% include content/plan-grid.md name="data-lakes" %}
5+
6+
The Segment Data Lakes sync history and health tabs generate real-time information about data syncs so you can monitor the health and performance of your data lakes. These tools provide monitoring and debugging capabilities within the Data Lakes UI, so you can identify and proactively address data sync or data pipeline failures.
7+
8+
> info ""
9+
> This feature is currently in Public Beta. Segment’s [First-Access and Beta terms](https://segment.com/legal/first-access-beta-preview/) govern the use of this feature.
10+
11+
## Sync History
12+
The 'Sync History' table shows detailed information about the latest 100 syncs to the data lake. The table includes the following fields:
13+
* **Sync status:** The status of the sync: either 'Success,' indicating that all rows synced correctly, 'Partial Success,' indicating that some rows synced correctly, or 'Failed,' indicating that no rows synced correctly
14+
* **Start time:** The time the sync began
15+
* **Duration:** How long the sync took to complete
16+
* **Synced rows:** The number of rows that synced to the data lake
17+
* **Notices:** Any notes or warnings about the sync
18+
19+
Selecting a row in the Sync History table opens a sidebar showing the number of rows from each collection that synced.
20+
21+
To access the Sync History page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes Settings page, select the **Sync History** tab.
22+
23+
## Health
24+
The health tab provides an overview of the rows that synced to your data lake both today and each day for the last 30 days.
25+
26+
The bar chart, 'Daily Synced Rows,' shows an overview of the rows synced for each of the last 30 days. Hovering over a date shows the number of rows that were synced for that day. Selecting a date from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection synced, and the percentage of all synced rows from each collection .
27+
28+
The Daily Row Volume table contains the following information:
29+
* **Collections:** The name of each collection of properties synced to the data lake
30+
* **Rows:** The number of rows synced from each collection
31+
* **% of Total:** The percentage of the total number of rows synced that each collection represents
32+
33+
Above the Daily Row Volume table is an overview of the total syncs for the current day, showing the number of rows synced, the number of collections that synced, and the current date.
34+
35+
To access the Sync history page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes settings page, select the **Health** tab.
36+
37+
## Data Lakes Reports FAQ
38+
##### How long is a data point available?
39+
The health tab shows an aggregate view of the last 30 days worth of data, while the sync history retains the last 100 syncs.
40+
41+
##### How do sync history and health compare?
42+
The sync history feature shows detailed information about the most recent 100 syncs to a data lake, while the health tab shows just the number of rows synced to the data lake over the last 30 days.
43+
44+
##### What timezone is the time and date information in?
45+
All dates and times on the sync history and health pages are in the user's local time.
46+
47+
##### When does the data update?
48+
The sync data for both reports updates in real time.
49+
50+
##### When do syncs occur?
51+
Syncs occur approximately every two hours. Users cannot choose how frequently the data lake syncs.

0 commit comments

Comments
 (0)