Simplify XMPP element metrics #4520
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4520 +/- ##
==========================================
+ Coverage 85.86% 85.88% +0.02%
==========================================
Files 565 565
Lines 33803 33795 -8
==========================================
Hits 29025 29025
+ Misses 4778 4770 -8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
9923835 to
84d7d06
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
84d7d06 to
9d1a993
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
9d1a993 to
4540c4f
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
4540c4f to
f39f613
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
f39f613 to
262ddac
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
elasticsearch_and_cassandra_27 / elasticsearch_and_cassandra_mnesia / 7da5aab small_tests_27 / small_tests / 7da5aab small_tests_26 / small_tests / 7da5aab small_tests_27_arm64 / small_tests / 7da5aab ldap_mnesia_26 / ldap_mnesia / 7da5aab ldap_mnesia_27 / ldap_mnesia / 7da5aab dynamic_domains_mysql_redis_27 / mysql_redis / 7da5aab dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / 7da5aab internal_mnesia_27 / internal_mnesia / 7da5aab dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 7da5aab pgsql_cets_27 / pgsql_cets / 7da5aab dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 7da5aab cockroachdb_cets_27 / cockroachdb_cets / 7da5aab pubsub_SUITE:dag+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3693@localhost/res1">>,
escalus_tcp,<0.117971.0>,
[{event_manager,<0.117959.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3693">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.117959.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3693">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3693">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3693">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"8f2d3c009a440dd0">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pubsub_SUITE:dag+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3696@localhost/res1">>,
escalus_tcp,<0.118048.0>,
[{event_manager,<0.118036.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3696">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.118036.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3696">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3696">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3696">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"f8e4777a9e2a2d15">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pubsub_SUITE:dag+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3700@localhost/res1">>,
escalus_tcp,<0.118126.0>,
[{event_manager,<0.118122.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3700">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.118122.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3700">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3700">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3700">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"2afee07a8376381d">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pubsub_SUITE:dag+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3702@localhost/res1">>,
escalus_tcp,<0.118201.0>,
[{event_manager,<0.118190.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3702">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.118190.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3702">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3702">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3702">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"a255f4cd8970e22f">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pubsub_SUITE:tree+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3758@localhost/res1">>,
escalus_tcp,<0.119488.0>,
[{event_manager,<0.119483.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3758">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.119483.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3758">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3758">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3758">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"3ac2dddbffbaf4a7">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...mysql_redis_27 / mysql_redis / 7da5aab mssql_mnesia_27 / odbc_mssql_mnesia / 7da5aab pgsql_mnesia_26 / pgsql_mnesia / 7da5aab |
The new module has two functions:
- execute_element_event/5 for incoming/outgoing XML elements
- instrumentation/1 for declaration of events for elements and data
Element events are simplified to xmpp_element_in/out with the following labels:
- connection_type: c2s, s2s or component
- host_type: it can be an empty binary if no host type is known,
e.g. for components, or for stream start errors.
* for exometer, empty host type is translated to 'global';
* prometheus treats empty string as no label, so no translation is needed.
Host type is always undefined for components.
It is now easier to collect all XMPP metrics (not only c2s), and the total traffic is a better estimation of system load, e.g. someone might only use components, and no c2s.
Also, refactor helpers to be more concise.
There is need to check for the empty host type in the labels, because components don't have host types assigned to them.
Replace 'global' with host type when necessary. This is needed, because all XML elements are reported per host type if the host type is known.
chrzaszcz
force-pushed
the
simplify-element-metrics
branch
from
7da5aab to
a631cef
Compare
|
elasticsearch_and_cassandra_27 / elasticsearch_and_cassandra_mnesia / a631cef small_tests_26 / small_tests / a631cef small_tests_27 / small_tests / a631cef small_tests_27_arm64 / small_tests / a631cef ldap_mnesia_26 / ldap_mnesia / a631cef ldap_mnesia_27 / ldap_mnesia / a631cef dynamic_domains_mysql_redis_27 / mysql_redis / a631cef internal_mnesia_27 / internal_mnesia / a631cef dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / a631cef dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / a631cef pgsql_cets_27 / pgsql_cets / a631cef dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / a631cef mysql_redis_27 / mysql_redis / a631cef cockroachdb_cets_27 / cockroachdb_cets / a631cef pubsub_SUITE:tree+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3845@localhost/res1">>,
escalus_tcp,<0.119354.0>,
[{event_manager,<0.119352.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3845">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.119352.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3845">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3845">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3845">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"47726d6b589f39f0">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pubsub_SUITE:tree+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3848@localhost/res1">>,
escalus_tcp,<0.119433.0>,
[{event_manager,<0.119429.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3848">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.119429.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3848">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3848">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3848">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"275b9a097e49e92f">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,434}]},
{pubsub_tools,receive_and_c...pgsql_mnesia_26 / pgsql_mnesia / a631cef pgsql_mnesia_27 / pgsql_mnesia / a631cef mssql_mnesia_27 / odbc_mssql_mnesia / a631cef |
The main goal is to simplify and unify events for XMPP (XML) elements. The main benefit is that unified metrics become easier to understand, maintain, test and document.
Before, there would be pairs of events like
c2s_element_inandxmpp_element_size_in. After, there is only one event:xmpp_element_in. The complete list is shown below:c2s_element_in(labels:host_type) andxmpp_element_size_in(labels:connection_type=c2s) are merged toxmpp_element_in(labels:host_type,connection_type=c2s).c2s_element_out(labels:host_type) andxmpp_element_size_out(labels:connection_type=c2s) are merged toxmpp_element_out(labels:host_type,connection_type=c2s).s2s_element_inandxmpp_element_size_in(labels:connection_type=s2s) are merged toxmpp_element_in(labels:host_type,connection_type=s2s).s2s_element_outandxmpp_element_size_out(labels:connection_type=s2s) are merged toxmpp_element_out(labels:host_type,connection_type=s2s).component_element_inandxmpp_element_size_in(labels:connection_type=component) are merged toxmpp_element_in(labels:host_type="",connection_type=component).component_element_outandxmpp_element_size_out(labels:connection_type=component) are merged toxmpp_element_out(labels:host_type="",connection_type=component).Notes:
c2s_element_*were missing if host type was unknown (e.g. in case of early stream errors). Now it is no longer the case.globalas the prefix in such cases.Left for the subsequent PR's: