Skip to content

Commit abc1241

Browse files
cyberkovcyberkov
committed
Add connector support to OpenTelemetry Collector
- Introduced `otelcol::connector` defined type for connecting pipelines. - Updated `otelcol` class to accept connectors configuration. - Added example for using connectors in `connector_example.pp`. - Created integration and unit tests for connector functionality. - Enhanced documentation in `REFERENCE.md` for connectors.
1 parent 0bd8488 commit abc1241

7 files changed

Lines changed: 340 additions & 0 deletions

File tree

‎REFERENCE.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020

2121
#### Public Defined types
2222

23+
* [`otelcol::connector`](#otelcol--connector): Add a connector to the OpenTelemetry Collector configuration
2324
* [`otelcol::exporter`](#otelcol--exporter): Define a OpenTelemetry Collector exporter
2425
* [`otelcol::extension`](#otelcol--extension): Add an extension to the OpenTelemetry Collector configuration
2526
* [`otelcol::pipeline`](#otelcol--pipeline): Add a pipeline to the OpenTelemetry Collector configuration
@@ -63,6 +64,7 @@ The following parameters are available in the `otelcol` class:
6364
* [`receivers`](#-otelcol--receivers)
6465
* [`processors`](#-otelcol--processors)
6566
* [`exporters`](#-otelcol--exporters)
67+
* [`connectors`](#-otelcol--connectors)
6668
* [`pipelines`](#-otelcol--pipelines)
6769
* [`extensions`](#-otelcol--extensions)
6870
* [`log_options`](#-otelcol--log_options)
@@ -191,6 +193,14 @@ Hash for exporters config
191193

192194
Default value: `{}`
193195

196+
##### <a name="-otelcol--connectors"></a>`connectors`
197+
198+
Data type: `Hash[String, Hash]`
199+
200+
Hash for connectors config
201+
202+
Default value: `{}`
203+
194204
##### <a name="-otelcol--pipelines"></a>`pipelines`
195205

196206
Data type: `Hash[String, Hash]`
@@ -330,6 +340,74 @@ Path to archive without filetype extension
330340

331341
## Defined types
332342

343+
### <a name="otelcol--connector"></a>`otelcol::connector`
344+
345+
Connectors are used to connect two pipelines. A connector acts as both an exporter
346+
(at the end of one pipeline) and a receiver (at the start of another pipeline).
347+
348+
#### Examples
349+
350+
##### basic connector
351+
352+
```puppet
353+
otelcol::connector { 'namevar': }
354+
```
355+
356+
##### Define a connector that converts traces to metrics
357+
358+
```puppet
359+
otelcol::connector { 'count':
360+
exporter_pipelines => ['traces'],
361+
receiver_pipelines => ['metrics'],
362+
}
363+
```
364+
365+
#### Parameters
366+
367+
The following parameters are available in the `otelcol::connector` defined type:
368+
369+
* [`name`](#-otelcol--connector--name)
370+
* [`config`](#-otelcol--connector--config)
371+
* [`order`](#-otelcol--connector--order)
372+
* [`exporter_pipelines`](#-otelcol--connector--exporter_pipelines)
373+
* [`receiver_pipelines`](#-otelcol--connector--receiver_pipelines)
374+
375+
##### <a name="-otelcol--connector--name"></a>`name`
376+
377+
The name of the connector
378+
379+
##### <a name="-otelcol--connector--config"></a>`config`
380+
381+
Data type: `Hash`
382+
383+
The configuration of the connector
384+
385+
Default value: `{}`
386+
387+
##### <a name="-otelcol--connector--order"></a>`order`
388+
389+
Data type: `Integer[0,999]`
390+
391+
The order of the connector
392+
393+
Default value: `0`
394+
395+
##### <a name="-otelcol--connector--exporter_pipelines"></a>`exporter_pipelines`
396+
397+
Data type: `Array[String[1]]`
398+
399+
The pipelines where the connector is used as an exporter (data flows out)
400+
401+
Default value: `[]`
402+
403+
##### <a name="-otelcol--connector--receiver_pipelines"></a>`receiver_pipelines`
404+
405+
Data type: `Array[String[1]]`
406+
407+
The pipelines where the connector is used as a receiver (data flows in)
408+
409+
Default value: `[]`
410+
333411
### <a name="otelcol--exporter"></a>`otelcol::exporter`
334412

335413
Create a OpenTelemetry Collector exporter in the configuration file.

‎examples/connector_example.pp‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Example using connectors to connect two pipelines
2+
# Connectors are used to connect two pipelines together in OpenTelemetry Collector
3+
# This example demonstrates the count connector converting traces to metrics
4+
5+
# Define a receiver for the traces pipeline
6+
otelcol::receiver { 'foo':
7+
config => {
8+
'protocols' => {
9+
'grpc' => { 'endpoint' => 'localhost:4317' },
10+
},
11+
},
12+
pipelines => ['traces'],
13+
}
14+
15+
# Define a connector that counts traces and outputs metrics
16+
# It acts as an exporter in the traces pipeline and a receiver in the metrics pipeline
17+
otelcol::connector { 'count':
18+
config => {},
19+
exporter_pipelines => ['traces'],
20+
receiver_pipelines => ['metrics'],
21+
}
22+
23+
# Define an exporter for the traces pipeline
24+
otelcol::exporter { 'bar':
25+
config => {
26+
'endpoint' => 'localhost:9090',
27+
},
28+
pipelines => ['metrics'],
29+
}
30+
31+
class { 'otelcol':
32+
manage_archive => true,
33+
}

‎manifests/config.pp‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,15 @@
102102
}
103103
}
104104

105+
$otelcol::connectors.each |String $rname, Hash $rvalue| {
106+
if $rvalue['config'] =~ Hash {
107+
ensure_resource('otelcol::connector', $rname, $rvalue)
108+
}
109+
else {
110+
ensure_resource('otelcol::connector', $rname, { 'config' => $rvalue })
111+
}
112+
}
113+
105114
$otelcol::pipelines.each |String $rname, Hash $rvalue| {
106115
if $rvalue['config'] =~ Hash {
107116
ensure_resource('otelcol::pipeline', $rname, $rvalue)

‎manifests/connector.pp‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# @summary Add a connector to the OpenTelemetry Collector configuration
2+
#
3+
# Connectors are used to connect two pipelines. A connector acts as both an exporter
4+
# (at the end of one pipeline) and a receiver (at the start of another pipeline).
5+
#
6+
# @param name
7+
# The name of the connector
8+
# @param config
9+
# The configuration of the connector
10+
# @param order
11+
# The order of the connector
12+
# @param exporter_pipelines
13+
# The pipelines where the connector is used as an exporter (data flows out)
14+
# @param receiver_pipelines
15+
# The pipelines where the connector is used as a receiver (data flows in)
16+
# @example basic connector
17+
# otelcol::connector { 'namevar': }
18+
# @example Define a connector that converts traces to metrics
19+
# otelcol::connector { 'count':
20+
# exporter_pipelines => ['traces'],
21+
# receiver_pipelines => ['metrics'],
22+
# }
23+
define otelcol::connector (
24+
Hash $config = {},
25+
Integer[0,999] $order = 0,
26+
Array[String[1]] $exporter_pipelines = [],
27+
Array[String[1]] $receiver_pipelines = [],
28+
) {
29+
$real_order = 1500+$order
30+
31+
# Define the connector in the connectors section
32+
$component = {
33+
'connectors' => {
34+
$name => $config,
35+
},
36+
}
37+
concat::fragment { "otelcol-config-connectors-${name}" :
38+
target => 'otelcol-config',
39+
order => $real_order,
40+
content => stdlib::to_yaml($component),
41+
}
42+
43+
# Add connector as exporter to specified pipelines
44+
$exporter_pipelines.each |String $pipeline| {
45+
$exporter_component = {
46+
'service' => {
47+
'pipelines' => {
48+
$pipeline => {
49+
'exporters' => [$name],
50+
},
51+
},
52+
},
53+
}
54+
concat::fragment { "otelcol-config-connector-${name}-exporter-${pipeline}" :
55+
target => 'otelcol-config',
56+
order => $real_order,
57+
content => stdlib::to_yaml($exporter_component),
58+
}
59+
}
60+
61+
# Add connector as receiver to specified pipelines
62+
$receiver_pipelines.each |String $pipeline| {
63+
$receiver_component = {
64+
'service' => {
65+
'pipelines' => {
66+
$pipeline => {
67+
'receivers' => [$name],
68+
},
69+
},
70+
},
71+
}
72+
concat::fragment { "otelcol-config-connector-${name}-receiver-${pipeline}" :
73+
target => 'otelcol-config',
74+
order => $real_order,
75+
content => stdlib::to_yaml($receiver_component),
76+
}
77+
}
78+
}

‎manifests/init.pp‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@
2929
# Hash for processors config
3030
# @param exporters
3131
# Hash for exporters config
32+
# @param connectors
33+
# Hash for connectors config
3234
# @param pipelines
3335
# Hash for pipelines config
3436
# @param extensions
@@ -70,6 +72,7 @@
7072
Hash[String, Hash] $receivers = {},
7173
Hash[String, Hash] $processors = {},
7274
Hash[String, Hash] $exporters = {},
75+
Hash[String, Hash] $connectors = {},
7376
Hash[String, Hash] $pipelines = {},
7477
Hash[String, Hash] $extensions = {},
7578
Variant[Hash,String[1]] $log_options = {},

‎spec/classes/connector_integration_spec.rb‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# frozen_string_literal: true
2+
3+
require 'spec_helper'
4+
5+
describe 'otelcol' do
6+
let(:pre_condition) do
7+
<<-PUPPET
8+
otelcol::receiver { 'foo':
9+
config => {},
10+
pipelines => ['traces'],
11+
}
12+
13+
otelcol::exporter { 'bar':
14+
config => {},
15+
pipelines => ['metrics'],
16+
}
17+
18+
otelcol::connector { 'count':
19+
config => {},
20+
exporter_pipelines => ['traces'],
21+
receiver_pipelines => ['metrics'],
22+
}
23+
PUPPET
24+
end
25+
26+
on_supported_os.each do |os, os_facts|
27+
context "on #{os} with connector pipeline" do
28+
let(:facts) { os_facts }
29+
let(:params) do
30+
{
31+
'manage_archive' => false,
32+
}
33+
end
34+
35+
it { is_expected.to compile.with_all_deps }
36+
37+
# Verify connector is defined in connectors section
38+
it {
39+
is_expected.to contain_concat__fragment('otelcol-config-connectors-count')
40+
}
41+
42+
# Verify connector is used as exporter in traces pipeline
43+
it {
44+
is_expected.to contain_concat__fragment('otelcol-config-connector-count-exporter-traces')
45+
}
46+
47+
# Verify connector is used as receiver in metrics pipeline
48+
it {
49+
is_expected.to contain_concat__fragment('otelcol-config-connector-count-receiver-metrics')
50+
}
51+
52+
# Verify receiver is in traces pipeline
53+
it {
54+
is_expected.to contain_concat__fragment('otelcol-config-receivers-foo-traces')
55+
}
56+
57+
# Verify exporter is in metrics pipeline
58+
it {
59+
is_expected.to contain_concat__fragment('otelcol-config-exporters-bar-metrics')
60+
}
61+
end
62+
end
63+
end

‎spec/defines/connector_spec.rb‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# frozen_string_literal: true
2+
3+
require 'spec_helper'
4+
5+
describe 'otelcol::connector' do
6+
let(:title) { 'count' }
7+
let(:params) do
8+
{
9+
'config' => {
10+
'key' => 'value',
11+
},
12+
}
13+
end
14+
15+
on_supported_os.each do |os, os_facts|
16+
context "on #{os}" do
17+
let(:facts) { os_facts }
18+
19+
it { is_expected.to compile }
20+
21+
it {
22+
is_expected.to contain_concat__fragment('otelcol-config-connectors-count').with({
23+
'order' => 1500,
24+
'target' => 'otelcol-config',
25+
})
26+
}
27+
28+
context 'with exporter and receiver pipelines' do
29+
let :params do
30+
super().merge({
31+
'exporter_pipelines' => ['traces'],
32+
'receiver_pipelines' => ['metrics'],
33+
})
34+
end
35+
36+
it { is_expected.to compile }
37+
38+
it {
39+
is_expected.to contain_concat__fragment('otelcol-config-connectors-count').with({
40+
'order' => 1500,
41+
'target' => 'otelcol-config',
42+
})
43+
}
44+
45+
it {
46+
is_expected.to contain_concat__fragment('otelcol-config-connector-count-exporter-traces').with({
47+
'order' => 1500,
48+
'target' => 'otelcol-config',
49+
})
50+
}
51+
52+
it {
53+
is_expected.to contain_concat__fragment('otelcol-config-connector-count-receiver-metrics').with({
54+
'order' => 1500,
55+
'target' => 'otelcol-config',
56+
})
57+
}
58+
end
59+
60+
context 'with order' do
61+
let :params do
62+
super().merge({ 'order' => 1 })
63+
end
64+
65+
it { is_expected.to compile }
66+
67+
it {
68+
is_expected.to contain_concat__fragment('otelcol-config-connectors-count').with({
69+
'order' => 1501,
70+
'target' => 'otelcol-config',
71+
})
72+
}
73+
end
74+
end
75+
end
76+
end

0 commit comments

Comments
 (0)