Skip to content

Commit 6ce3249

Browse files
julienduchesnejulienduchesne
authored
Merge pull request #792 from grafana/julienduchesne/upstream
Sync upstream to current HEAD at `e2f037e554a8a336ca4487324dd4ceee893fda0a`
2 parents 0a0a416 + dbdc94f commit 6ce3249

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+3636
-1764
lines changed

‎.github/dependabot.yml

+14
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,20 @@ updates:
2020
open-pull-requests-limit: 0
2121
- package-ecosystem: "npm"
2222
directory: "/web/ui"
23+
labels:
24+
- dependencies
25+
- javascript
26+
- manteen-ui
27+
schedule:
28+
interval: "monthly"
29+
open-pull-requests-limit: 20
30+
# Old react-app packages.
31+
- package-ecosystem: "npm"
32+
directory: "/web/ui/react-app"
33+
labels:
34+
- dependencies
35+
- javascript
36+
- old-react-ui
2337
schedule:
2438
interval: "monthly"
2539
open-pull-requests-limit: 0

‎RELEASE.md

+9-55
Original file line numberDiff line numberDiff line change
@@ -5,61 +5,15 @@ This page describes the release process and the currently planned schedule for u
55
## Release schedule
66

77
Release cadence of first pre-releases being cut is 6 weeks.
8+
Please see [the v2.55 RELEASE.md](https://github.com/prometheus/prometheus/blob/release-2.55/RELEASE.md) for the v2 release series schedule.
89

9-
| release series | date of first pre-release (year-month-day) | release shepherd |
10-
|----------------|--------------------------------------------|---------------------------------------------|
11-
| v2.4 | 2018-09-06 | Goutham Veeramachaneni (GitHub: @gouthamve) |
12-
| v2.5 | 2018-10-24 | Frederic Branczyk (GitHub: @brancz) |
13-
| v2.6 | 2018-12-05 | Simon Pasquier (GitHub: @simonpasquier) |
14-
| v2.7 | 2019-01-16 | Goutham Veeramachaneni (GitHub: @gouthamve) |
15-
| v2.8 | 2019-02-27 | Ganesh Vernekar (GitHub: @codesome) |
16-
| v2.9 | 2019-04-10 | Brian Brazil (GitHub: @brian-brazil) |
17-
| v2.10 | 2019-05-22 | Björn Rabenstein (GitHub: @beorn7) |
18-
| v2.11 | 2019-07-03 | Frederic Branczyk (GitHub: @brancz) |
19-
| v2.12 | 2019-08-14 | Julius Volz (GitHub: @juliusv) |
20-
| v2.13 | 2019-09-25 | Krasi Georgiev (GitHub: @krasi-georgiev) |
21-
| v2.14 | 2019-11-06 | Chris Marchbanks (GitHub: @csmarchbanks) |
22-
| v2.15 | 2019-12-18 | Bartek Plotka (GitHub: @bwplotka) |
23-
| v2.16 | 2020-01-29 | Callum Styan (GitHub: @cstyan) |
24-
| v2.17 | 2020-03-11 | Julien Pivotto (GitHub: @roidelapluie) |
25-
| v2.18 | 2020-04-22 | Bartek Plotka (GitHub: @bwplotka) |
26-
| v2.19 | 2020-06-03 | Ganesh Vernekar (GitHub: @codesome) |
27-
| v2.20 | 2020-07-15 | Björn Rabenstein (GitHub: @beorn7) |
28-
| v2.21 | 2020-08-26 | Julien Pivotto (GitHub: @roidelapluie) |
29-
| v2.22 | 2020-10-07 | Frederic Branczyk (GitHub: @brancz) |
30-
| v2.23 | 2020-11-18 | Ganesh Vernekar (GitHub: @codesome) |
31-
| v2.24 | 2020-12-30 | Björn Rabenstein (GitHub: @beorn7) |
32-
| v2.25 | 2021-02-10 | Julien Pivotto (GitHub: @roidelapluie) |
33-
| v2.26 | 2021-03-24 | Bartek Plotka (GitHub: @bwplotka) |
34-
| v2.27 | 2021-05-05 | Chris Marchbanks (GitHub: @csmarchbanks) |
35-
| v2.28 | 2021-06-16 | Julius Volz (GitHub: @juliusv) |
36-
| v2.29 | 2021-07-28 | Frederic Branczyk (GitHub: @brancz) |
37-
| v2.30 | 2021-09-08 | Ganesh Vernekar (GitHub: @codesome) |
38-
| v2.31 | 2021-10-20 | Julien Pivotto (GitHub: @roidelapluie) |
39-
| v2.32 | 2021-12-01 | Julius Volz (GitHub: @juliusv) |
40-
| v2.33 | 2022-01-12 | Björn Rabenstein (GitHub: @beorn7) |
41-
| v2.34 | 2022-02-23 | Chris Marchbanks (GitHub: @csmarchbanks) |
42-
| v2.35 | 2022-04-06 | Augustin Husson (GitHub: @nexucis) |
43-
| v2.36 | 2022-05-18 | Matthias Loibl (GitHub: @metalmatze) |
44-
| v2.37 LTS | 2022-06-29 | Julien Pivotto (GitHub: @roidelapluie) |
45-
| v2.38 | 2022-08-10 | Julius Volz (GitHub: @juliusv) |
46-
| v2.39 | 2022-09-21 | Ganesh Vernekar (GitHub: @codesome) |
47-
| v2.40 | 2022-11-02 | Ganesh Vernekar (GitHub: @codesome) |
48-
| v2.41 | 2022-12-14 | Julien Pivotto (GitHub: @roidelapluie) |
49-
| v2.42 | 2023-01-25 | Kemal Akkoyun (GitHub: @kakkoyun) |
50-
| v2.43 | 2023-03-08 | Julien Pivotto (GitHub: @roidelapluie) |
51-
| v2.44 | 2023-04-19 | Bryan Boreham (GitHub: @bboreham) |
52-
| v2.45 LTS | 2023-05-31 | Jesus Vazquez (Github: @jesusvazquez) |
53-
| v2.46 | 2023-07-12 | Julien Pivotto (GitHub: @roidelapluie) |
54-
| v2.47 | 2023-08-23 | Bryan Boreham (GitHub: @bboreham) |
55-
| v2.48 | 2023-10-04 | Levi Harrison (GitHub: @LeviHarrison) |
56-
| v2.49 | 2023-12-05 | Bartek Plotka (GitHub: @bwplotka) |
57-
| v2.50 | 2024-01-16 | Augustin Husson (GitHub: @nexucis) |
58-
| v2.51 | 2024-03-07 | Bryan Boreham (GitHub: @bboreham) |
59-
| v2.52 | 2024-04-22 | Arthur Silva Sens (GitHub: @ArthurSens) |
60-
| v2.53 LTS | 2024-06-03 | George Krajcsovits (GitHub: @krajorama) |
61-
| v2.54 | 2024-07-17 | Bryan Boreham (GitHub: @bboreham) |
62-
| v2.55 | 2024-09-17 | Bryan Boreham (GitHub: @bboreham) |
10+
| release series | date of first pre-release (year-month-day) | release shepherd |
11+
|----------------|--------------------------------------------|-----------------------------------|
12+
| v3.0 | 2024-11-14 | Jan Fajerski (GitHub: @jan--f) |
13+
| v3.1 | 2024-12-17 | Bryan Boreham (GitHub: @bboreham) |
14+
| v3.2 | 2025-01-28 | Jan Fajerski (GitHub: @jan--f) |
15+
| v3.3 | 2025-03-11 | Ayoub Mrini (Github: @machine424) |
16+
| v3.4 | 2025-04-22 | **volunteer welcome** |
6317

6418
If you are interested in volunteering please create a pull request against the [prometheus/prometheus](https://github.com/prometheus/prometheus) repository and propose yourself for the release series of your choice.
6519

@@ -204,7 +158,7 @@ Then release with `git tag-release`.
204158

205159
Signing a tag with a GPG key is appreciated, but in case you can't add a GPG key to your Github account using the following [procedure](https://help.github.com/articles/generating-a-gpg-key/), you can replace the `-s` flag by `-a` flag of the `git tag` command to only annotate the tag without signing.
206160

207-
Once a tag is created, the release process through CircleCI will be triggered for this tag and Circle CI will draft the GitHub release using the `prombot` account.
161+
Once a tag is created, the release process through Github Actions will be triggered for this tag and Github Actions will draft the GitHub release using the `prombot` account.
208162

209163
Finally, wait for the build step for the tag to finish. The point here is to wait for tarballs to be uploaded to the Github release and the container images to be pushed to the Docker Hub and Quay.io. Once that has happened, click _Publish release_, which will make the release publicly visible and create a GitHub notification.
210164
**Note:** for a release candidate version ensure the _This is a pre-release_ box is checked when drafting the release in the Github UI. The CI job should take care of this but it's a good idea to double check before clicking _Publish release_.`

‎cmd/prometheus/main.go

+3-1
Original file line numberDiff line numberDiff line change
@@ -594,12 +594,14 @@ func main() {
594594
logger.Error(fmt.Sprintf("Error loading config (--config.file=%s)", cfg.configFile), "file", absPath, "err", err)
595595
os.Exit(2)
596596
}
597+
// Get scrape configs to validate dynamically loaded scrape_config_files.
598+
// They can change over time, but do the extra validation on startup for better experience.
597599
if _, err := cfgFile.GetScrapeConfigs(); err != nil {
598600
absPath, pathErr := filepath.Abs(cfg.configFile)
599601
if pathErr != nil {
600602
absPath = cfg.configFile
601603
}
602-
logger.Error(fmt.Sprintf("Error loading scrape config files from config (--config.file=%q)", cfg.configFile), "file", absPath, "err", err)
604+
logger.Error(fmt.Sprintf("Error loading dynamic scrape config files from config (--config.file=%q)", cfg.configFile), "file", absPath, "err", err)
603605
os.Exit(2)
604606
}
605607
if cfg.tsdb.EnableExemplarStorage {

‎config/config.go

+22-15
Original file line numberDiff line numberDiff line change
@@ -117,11 +117,12 @@ func Load(s string, logger *slog.Logger) (*Config, error) {
117117
default:
118118
return nil, fmt.Errorf("unsupported OTLP translation strategy %q", cfg.OTLPConfig.TranslationStrategy)
119119
}
120-
120+
cfg.loaded = true
121121
return cfg, nil
122122
}
123123

124-
// LoadFile parses the given YAML file into a Config.
124+
// LoadFile parses and validates the given YAML file into a read-only Config.
125+
// Callers should never write to or shallow copy the returned Config.
125126
func LoadFile(filename string, agentMode bool, logger *slog.Logger) (*Config, error) {
126127
content, err := os.ReadFile(filename)
127128
if err != nil {
@@ -270,9 +271,12 @@ type Config struct {
270271
RemoteWriteConfigs []*RemoteWriteConfig `yaml:"remote_write,omitempty"`
271272
RemoteReadConfigs []*RemoteReadConfig `yaml:"remote_read,omitempty"`
272273
OTLPConfig OTLPConfig `yaml:"otlp,omitempty"`
274+
275+
loaded bool // Certain methods require configuration to use Load validation.
273276
}
274277

275278
// SetDirectory joins any relative file paths with dir.
279+
// This method writes to config, and it's not concurrency safe.
276280
func (c *Config) SetDirectory(dir string) {
277281
c.GlobalConfig.SetDirectory(dir)
278282
c.AlertingConfig.SetDirectory(dir)
@@ -302,24 +306,26 @@ func (c Config) String() string {
302306
return string(b)
303307
}
304308

305-
// GetScrapeConfigs returns the scrape configurations.
309+
// GetScrapeConfigs returns the read-only, validated scrape configurations including
310+
// the ones from the scrape_config_files.
311+
// This method does not write to config, and it's concurrency safe (the pointer receiver is for efficiency).
312+
// This method also assumes the Config was created by Load or LoadFile function, it returns error
313+
// if it was not. We can't re-validate or apply globals here due to races,
314+
// read more https://github.com/prometheus/prometheus/issues/15538.
306315
func (c *Config) GetScrapeConfigs() ([]*ScrapeConfig, error) {
307-
scfgs := make([]*ScrapeConfig, len(c.ScrapeConfigs))
316+
if !c.loaded {
317+
// Programmatic error, we warn before more confusing errors would happen due to lack of the globalization.
318+
return nil, errors.New("scrape config cannot be fetched, main config was not validated and loaded correctly; should not happen")
319+
}
308320

321+
scfgs := make([]*ScrapeConfig, len(c.ScrapeConfigs))
309322
jobNames := map[string]string{}
310323
for i, scfg := range c.ScrapeConfigs {
311-
// We do these checks for library users that would not call validate in
312-
// Unmarshal.
313-
if err := scfg.Validate(c.GlobalConfig); err != nil {
314-
return nil, err
315-
}
316-
317-
if _, ok := jobNames[scfg.JobName]; ok {
318-
return nil, fmt.Errorf("found multiple scrape configs with job name %q", scfg.JobName)
319-
}
320324
jobNames[scfg.JobName] = "main config file"
321325
scfgs[i] = scfg
322326
}
327+
328+
// Re-read and validate the dynamic scrape config rules.
323329
for _, pat := range c.ScrapeConfigFiles {
324330
fs, err := filepath.Glob(pat)
325331
if err != nil {
@@ -355,6 +361,7 @@ func (c *Config) GetScrapeConfigs() ([]*ScrapeConfig, error) {
355361
}
356362

357363
// UnmarshalYAML implements the yaml.Unmarshaler interface.
364+
// NOTE: This method should not be used outside of this package. Use Load or LoadFile instead.
358365
func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
359366
*c = DefaultConfig
360367
// We want to set c to the defaults and then overwrite it with the input.
@@ -391,18 +398,18 @@ func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
391398
}
392399
}
393400

394-
// Do global overrides and validate unique names.
401+
// Do global overrides and validation.
395402
jobNames := map[string]struct{}{}
396403
for _, scfg := range c.ScrapeConfigs {
397404
if err := scfg.Validate(c.GlobalConfig); err != nil {
398405
return err
399406
}
400-
401407
if _, ok := jobNames[scfg.JobName]; ok {
402408
return fmt.Errorf("found multiple scrape configs with job name %q", scfg.JobName)
403409
}
404410
jobNames[scfg.JobName] = struct{}{}
405411
}
412+
406413
rwNames := map[string]struct{}{}
407414
for _, rwcfg := range c.RemoteWriteConfigs {
408415
if rwcfg == nil {

‎config/config_default_test.go

+2
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ package config
1818
const ruleFilesConfigFile = "testdata/rules_abs_path.good.yml"
1919

2020
var ruleFilesExpectedConf = &Config{
21+
loaded: true,
22+
2123
GlobalConfig: DefaultGlobalConfig,
2224
Runtime: DefaultRuntimeConfig,
2325
RuleFiles: []string{

‎config/config_test.go

+22-4
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ const (
8787
)
8888

8989
var expectedConf = &Config{
90+
loaded: true,
9091
GlobalConfig: GlobalConfig{
9192
ScrapeInterval: model.Duration(15 * time.Second),
9293
ScrapeTimeout: DefaultGlobalConfig.ScrapeTimeout,
@@ -1512,10 +1513,10 @@ func TestYAMLRoundtrip(t *testing.T) {
15121513
require.NoError(t, err)
15131514

15141515
out, err := yaml.Marshal(want)
1516+
require.NoError(t, err)
15151517

1518+
got, err := Load(string(out), promslog.NewNopLogger())
15161519
require.NoError(t, err)
1517-
got := &Config{}
1518-
require.NoError(t, yaml.UnmarshalStrict(out, got))
15191520

15201521
require.Equal(t, want, got)
15211522
}
@@ -1525,10 +1526,10 @@ func TestRemoteWriteRetryOnRateLimit(t *testing.T) {
15251526
require.NoError(t, err)
15261527

15271528
out, err := yaml.Marshal(want)
1529+
require.NoError(t, err)
15281530

1531+
got, err := Load(string(out), promslog.NewNopLogger())
15291532
require.NoError(t, err)
1530-
got := &Config{}
1531-
require.NoError(t, yaml.UnmarshalStrict(out, got))
15321533

15331534
require.True(t, got.RemoteWriteConfigs[0].QueueConfig.RetryOnRateLimit)
15341535
require.False(t, got.RemoteWriteConfigs[1].QueueConfig.RetryOnRateLimit)
@@ -2219,6 +2220,7 @@ func TestEmptyConfig(t *testing.T) {
22192220
c, err := Load("", promslog.NewNopLogger())
22202221
require.NoError(t, err)
22212222
exp := DefaultConfig
2223+
exp.loaded = true
22222224
require.Equal(t, exp, *c)
22232225
}
22242226

@@ -2268,6 +2270,7 @@ func TestEmptyGlobalBlock(t *testing.T) {
22682270
require.NoError(t, err)
22692271
exp := DefaultConfig
22702272
exp.Runtime = DefaultRuntimeConfig
2273+
exp.loaded = true
22712274
require.Equal(t, exp, *c)
22722275
}
22732276

@@ -2548,3 +2551,18 @@ func TestScrapeProtocolHeader(t *testing.T) {
25482551
})
25492552
}
25502553
}
2554+
2555+
// Regression test against https://github.com/prometheus/prometheus/issues/15538
2556+
func TestGetScrapeConfigs_Loaded(t *testing.T) {
2557+
t.Run("without load", func(t *testing.T) {
2558+
c := &Config{}
2559+
_, err := c.GetScrapeConfigs()
2560+
require.EqualError(t, err, "scrape config cannot be fetched, main config was not validated and loaded correctly; should not happen")
2561+
})
2562+
t.Run("with load", func(t *testing.T) {
2563+
c, err := Load("", promslog.NewNopLogger())
2564+
require.NoError(t, err)
2565+
_, err = c.GetScrapeConfigs()
2566+
require.NoError(t, err)
2567+
})
2568+
}

‎config/config_windows_test.go

+2
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ package config
1616
const ruleFilesConfigFile = "testdata/rules_abs_path_windows.good.yml"
1717

1818
var ruleFilesExpectedConf = &Config{
19+
loaded: true,
20+
1921
GlobalConfig: DefaultGlobalConfig,
2022
Runtime: DefaultRuntimeConfig,
2123
RuleFiles: []string{

‎docs/configuration/configuration.md

+7
Original file line numberDiff line numberDiff line change
@@ -676,6 +676,13 @@ http_headers:
676676

677677
Azure SD configurations allow retrieving scrape targets from Azure VMs.
678678

679+
The discovery requires at least the following permissions:
680+
681+
* `Microsoft.Compute/virtualMachines/read`: Required for VM discovery
682+
* `Microsoft.Network/networkInterfaces/read`: Required for VM discovery
683+
* `Microsoft.Compute/virtualMachineScaleSets/virtualMachines/read`: Required for scale set (VMSS) discovery
684+
* `Microsoft.Compute/virtualMachineScaleSets/virtualMachines/networkInterfaces/read`: Required for scale set (VMSS) discovery
685+
679686
The following meta labels are available on targets during [relabeling](#relabel_config):
680687

681688
* `__meta_azure_machine_id`: the machine ID

0 commit comments

Comments
 (0)