Created
July 25, 2023 15:37
-
-
Save pracucci/a5a390fe0cb3da5222208fffac92fcf7 to your computer and use it in GitHub Desktop.
Label names and values query implementation in Mimir (updated: 2023-07-25)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Get label names | |
- Ingester | |
- TSDB LabelNames() | |
- Head | |
- Without matchers: | |
- Read from MemPostings -> fast | |
- With matchers: | |
- PostingsForMatchers() (like any query) | |
- LabelNamesFor() | |
- Symbolise each series, iterate on the label names and put them into a map (to get the unique set) | |
- Block | |
- Without matchers: | |
- Read from in-memory postings map -> fast | |
- With matchers: | |
- PostingsForMatchers() (like any query) | |
- LabelNamesFor() | |
- Symbolise each series, iterate on the label names and put them into a map (to get the unique set) | |
- Store-gateway | |
- Long TTL cache since blocks are immutable | |
- Without matchers: | |
- Read from in-memory postings map -> fast (assuming the block has already been lazy-loaded) | |
- With matchers: | |
- Find matching series, symbolise series, iterate on the label names and put them into a map (to get the unique set) | |
### Label names cardinality | |
- Ingester (LabelNamesAndValues()) | |
- Query only the TSDB Head | |
- index.LabelNames(matchers...) | |
- For each label name: | |
- index.LabelValues(labelName, matchers...) | |
- Store-gateway not queried | |
### Get series by label matchers | |
- Ingester | |
- MetricsForLabelMatchers() | |
- For each matcher: | |
- Call q.Select() to find matching series (without fetching chunks) | |
- Store-gateway | |
- Series() with skipChunks=true | |
### Get label values | |
- Ingester | |
- tsdb.Querier().LabelValues(labelName, matchers...) | |
- Head and Block | |
- Without matchers: | |
- h.head.postings.LabelValues(name) | |
- Iterate all label values by label name equal to input (from MemPostings), put them into a map (to get the unique set) -> fast | |
- With matchers: | |
- PostingsForMatchers(r, matchers...) | |
- h.head.postings.LabelValues(name) | |
- For each label value returned by LabelValues(name): | |
- Call r.Postings(name, value) to get the list of postings for matching label name and value | |
- Call index.FindIntersectingPostings() to find the intersection between the result of PostingsForMatchers() and the candidate postings found calling Postings() for each name-value pair | |
- Store-gateway | |
- Long TTL cache since blocks are immutable | |
- Without matchers: | |
- Find all postings for the label name, querying the index | |
- With matchers: | |
- Find all postings for the label name, querying the index | |
- Either labelValuesFromSeries() or labelValuesFromPostings() | |
- labelValuesFromSeries() | |
- Run a normal Series() call and then build a map with label values (to get unique set) | |
- labelValuesFromPostings() | |
- Works similarly to TSDB | |
### Label values cardinality | |
- Ingester | |
- For each label name in the request | |
- Get the label values calling: idxReader.LabelValues(lblName, matchers...) | |
- For each label value: | |
- Call PostingsForMatchers(), then iterate to count the number of matching series | |
- The PostingsForMatchers function can be: | |
- in-memory: tsdb.PostingsForMatchers() | |
- active: activeseries.NewPostings() wrapping tsdb.PostingsForMatchers() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment