How Logit.io calculates daily index sizes and used volume statistics
Why is the index size sometimes less than that reported in Stack usage Statistics?
Daily log usage (Logs Sent) is based on the size of the primary data stored in that day's indexes. We measure it about every 10 minutes through the day and your daily usage is the highest measurement for that day (UTC).
Rolling out: OpenSearch regularly merges smaller index segments into larger ones in the background. While a merge is running, the new segment is written before the old ones are removed, so the store size can briefly look much bigger than your data. We're rolling out an update that leaves out data that's being merged when we measure your usage, so these short merge peaks no longer inflate your daily usage. Deleting data after it has been measured doesn't lower that day's usage.
Replicas aren't counted, only the primary data.
It's important to be aware that when viewing the Elasticsearch index
status (using /_cat/indices) this page reports by default in GiB (opens in a new tab)
and Logit reports usage in the more commonly used Gigabyte GB (opens in a new tab),
hence a difference when trying to compare the values.
You can always query the elasticsearch index api and return the
pri.store.size in bytes /_cat/indices?h=index,pri.store.size&bytes=b&v,
which can then be converted into GB's.
Are the elastalert_status indexes included in the daily usage statistics?
The elastalert_status indexes aren't included in the usage statistics and the cut of time for index size calculation is UTC 00:00. Indexes that are currently in use are normally slightly larger in size than older indexes, this is due to older indexes being compressed.