Skip to content

Commit 1f8bdbc

Browse files
committed
DOC-13588: Issues with streaming completed requests (#434)
* Fix property name * Add completed-stream-size to Configure Queries ToC * Vale updates * Update preview config
1 parent b53a894 commit 1f8bdbc

3 files changed

Lines changed: 56 additions & 52 deletions

File tree

modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc

Lines changed: 33 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,26 @@
11
= Manage and Monitor Queries
22
:example-caption!:
3-
:description: Monitoring and profiling {sqlpp} queries, query service nodes, and corresponding system resources is very important for smoother operational performance and efficiency of the system.
3+
:description: Monitoring and profiling {sqlpp} queries, Query Service nodes, and corresponding system resources is important for smoother operational performance and efficiency of the system.
44
:page-aliases: monitoring:monitoring-n1ql-query, manage:monitor/monitoring-n1ql-query
55

66
[abstract]
77
{description}
8-
In fact, often it is vital for diagnosing and troubleshooting issues such as query performance, resource bottlenecks, and overloading of various services.
8+
In fact, often it's vital for diagnosing and troubleshooting issues such as query performance, resource bottlenecks, and overloading of various services.
99

1010
include::ROOT:partial$component-signpost.adoc[]
1111

12-
System keyspaces provide various monitoring details and statistics about individual queries and the Query service.
12+
System keyspaces provide various monitoring details and statistics about individual queries and the Query Service.
1313
When running on a cluster with multiple query nodes, stats about all queries on all query nodes are collected in the Query management and monitoring system keyspaces.
1414

1515
For example, this can help identify:
1616

1717
* The top 10 slow or fast queries running on a particular query node or the cluster.
18-
* Resource usage statistics of the query service, or resources used for a particular query.
18+
* Resource usage statistics of the Query Service, or resources used for a particular query.
1919
* Details about the active, completed, and prepared queries.
2020
* Find long running queries that are running for more than 2 minutes.
2121
2222
These system keyspaces are transient in nature, and are not persisted to disk or permanent storage.
23-
Hence, the information in the keyspaces pertains to the current instantiation of the Query service.
23+
Hence, the information in the keyspaces pertains to the current instantiation of the Query Service.
2424

2525
You can access the Query management and monitoring system keyspaces using any of the following:
2626

@@ -33,7 +33,7 @@ Using {sqlpp} enables you to obtain further insights from the keyspaces.
3333
== Authentication and Client Privileges
3434

3535
Users must have the *Query System Catalog* role to access restricted system keyspaces.
36-
For more details about user roles, see xref:learn:security/authorization-overview.adoc[Authorization].
36+
For more information about user roles, see xref:learn:security/authorization-overview.adoc[Authorization].
3737

3838
== Examples on this Page
3939

@@ -47,7 +47,7 @@ In the REST API examples:
4747
== Monitor System Vitals
4848

4949
The `system:vitals` catalog provides data about the running state and health of the query nodes, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on.
50-
This information can be very useful to assess the current workload and performance characteristics of a query node.
50+
This information can be useful to assess the current workload and performance characteristics of a query node.
5151

5252
[#sys-vitals-get]
5353
=== Get System Vitals
@@ -475,7 +475,7 @@ Getting prepared statements, as described in <<sys-prepared-get>>, returns resul
475475
}
476476
----
477477
478-
In this example, the names of the prepared statements are identical, but they are associated with different query contexts.
478+
In this example, the names of the prepared statements are identical, but they're associated with different query contexts.
479479
480480
<.> The name of the prepared statement for the default query context
481481
<.> The name of the prepared statement showing the associated query context
@@ -524,7 +524,7 @@ SELECT * FROM system:completed_requests;
524524
--
525525
====
526526

527-
Note that the `completed` state means that the request was started and completed by the Query service, but it does not mean that it was necessarily successful.
527+
The `completed` state means that the request was started and completed by the Query Service, but it does not mean that it was necessarily successful.
528528
The request could have been successful, or completed with errors.
529529

530530
To find requests that completed successfully, search for completed requests whose `state` is `completed` and whose `errorCount` field has the value `0`.
@@ -610,7 +610,7 @@ DELETE FROM system:completed_requests WHERE requestTime LIKE "2015-09-09%";
610610
[[sys-completed-examples]]
611611
=== Completed Request Details
612612

613-
To try the examples in this section, first run a query which takes at least 1000ms (the default value of the `completed-threshold` query setting) to get registered in the `system:completed_requests` keyspace.
613+
To try the examples in this section, first run a query which takes at least 1000{nbsp}ms (the default value of the `completed-threshold` query setting) to get registered in the `system:completed_requests` keyspace.
614614

615615
.Run a long query
616616
====
@@ -720,7 +720,7 @@ curl $BASE_URL/admin/settings -u $USER:$PASSWORD \
720720
=== Logging Qualifiers
721721

722722
You can specify the following logging qualifiers.
723-
A completed request is logged if _any_ of the qualifiers are met (logical OR).
723+
A completed request is logged if any of the qualifiers are met (logical OR).
724724

725725
[horizontal]
726726
`threshold`:: The execution time threshold in milliseconds.
@@ -735,11 +735,12 @@ A completed request is logged if _any_ of the qualifiers are met (logical OR).
735735

736736
For full details, see xref:n1ql-rest-admin:index.adoc#Logging_Parameters[Logging Parameters].
737737

738-
The basic syntax adds a qualifier to the logging parameters, i.e. any existing qualifiers are not removed.
738+
The basic syntax adds a qualifier to the logging parameters.
739+
Any existing qualifiers are not removed.
739740
You can change the value of a logging qualifier by specifying the same qualifier again with a new value.
740741

741-
To add a new instance of an existing qualifier, use a plus sign (`+`) before the qualifier name, e.g. `+user`.
742-
To remove a qualifier, use a minus sign (`-`) before the qualifier name, e.g. `-user`.
742+
To add a new instance of an existing qualifier, use a plus sign (`+`) before the qualifier name, such as `+user`.
743+
To remove a qualifier, use a minus sign (`-`) before the qualifier name, such as `-user`.
743744

744745
For example, the following request will add user `simon` to those tracked, and remove error `12003`.
745746

@@ -790,7 +791,7 @@ You cannot add a new instance of an existing qualifier to a tagged set using a p
790791
For example, you cannot add a `user` qualifier to a tagged set that already contains a `user` qualifier.
791792
If you need to track two users with the same error, create two tagged sets, one per user.
792793

793-
You can remove a qualifier from a tagged set using a minus sign (`-`) before the qualifier name, e.g. `-user`.
794+
You can remove a qualifier from a tagged set using a minus sign (`-`) before the qualifier name, such as `-user`.
794795
When you remove the last qualifier from a tagged set, the tagged set is removed.
795796

796797
[NOTE]
@@ -807,7 +808,7 @@ In this case, completed requests are logged if they match any of the individual
807808
The [.param]`completed-threshold` field provides another way of specifying the `threshold` qualifier within the `completed` field.
808809

809810
This field sets the minimum request duration after which requests are added to the `system:completed_requests` catalog.
810-
The default value is 1000ms.
811+
The default value is 1000{nbsp}ms.
811812
Specify [.in]`0` to log all requests and [.in]`-1` to not log any requests to the keyspace.
812813

813814
To specify a different value, use:
@@ -841,13 +842,13 @@ curl $BASE_URL/admin/settings -u $USER:$PASSWORD \
841842

842843
In Couchbase Server 7.6.4 and later, you can stream completed requests to disk.
843844

844-
To enable completed request streaming, use the xref:n1ql:n1ql-rest-api/admin.adoc[Admin REST API] `/admin/settings` endpoint to specify the `completed_stream_size` property.
845+
To enable completed request streaming, use the xref:n1ql:n1ql-rest-api/admin.adoc[Admin REST API] `/admin/settings` endpoint to specify the `completed-stream-size` property.
845846

846847
[source,sh]
847848
----
848849
curl $BASE_URL/admin/settings -u $USER:$PASSWORD \
849850
-H 'Content-Type: application/json' \
850-
-d '{"completed_stream_size":500}'
851+
-d '{"completed-stream-size": 500}'
851852
----
852853

853854
This property is a file size in MiB.
@@ -857,25 +858,25 @@ When set to any size greater than `0`, completed requests are streamed to archiv
857858
The value of this property determines the size of the data to retain, per node.
858859
The configuration for completed requests determines which requests are saved.
859860

860-
NOTE: The additional processing required to save completed requests to disk may limit overall request throughput on a Query node, but typically only when every completed request is being recorded, and requests are very small or short-lived.
861-
The speed of the file system on which the server logs directory resides naturally affects the potential impact too.
861+
NOTE: The additional processing required to save completed requests to disk may limit overall request throughput on a Query node, but typically only when every completed request is being recorded, and requests are small or short-lived.
862+
The speed of the file system on which the server logs directory resides may affect the request throughput also.
862863

863864
[#sys-history-files]
864865
=== Archived Request Files
865866

866867
When streaming is enabled, completed requests are saved to GZIP archives with the prefix `local_request_log` in the Couchbase Server `logs` directory.
867868
Each saved GZIP archive file contains multiple JSON entries, one for each for each recorded completed request.
868869

869-
Couchbase Server writes multiple archive files in parallel, so whilst the order of requests in a file is sequential, a single given file may not contain a contiguous sequence of requests.
870+
Couchbase Server writes multiple archive files in parallel, so while the order of requests in a file is sequential, a single given file may not contain a contiguous sequence of requests.
870871

871-
When an archive file reaches or exceeds 100 MiB, it is finalized and saved to disk.
872+
When an archive file reaches or exceeds 100 MiB, it's finalized and saved to disk.
872873
This is not a hard limit -- entries are not truncated to adhere to it.
873874
Files may also be finalized with less content, if nothing has been written to them for an extended period.
874-
Files that are actively being written are not available for reading, and they don't count towards the configured size limit until they're finalized.
875+
Files that are actively being written are not available for reading, and they do not count towards the configured size limit until they're finalized.
875876

876877
Couchbase Server tries to manage and retain archive files such that the total disk space used by the files is within the specified limit for the node.
877878
When the specified limit is reached, older files are removed as necessary to make space for newly finalized files.
878-
When a file is removed, it isn't guaranteed that only the oldest requests are evicted, given that Couchbase Server writes to multiple archive files in parallel.
879+
When a file is removed, it's not guaranteed that only the oldest requests are evicted, given that Couchbase Server writes to multiple archive files in parallel.
879880

880881
[#sys-history-view]
881882
=== View Archived Requests
@@ -907,7 +908,7 @@ SELECT * FROM system:completed_requests_history;
907908
--
908909
====
909910

910-
The `system:completed_requests_history` keyspace is provided for {sqlpp} access to the archived files, but as they are external GZIP archives performance is restricted, particularly with large histories on clusters with multiple Query service nodes.
911+
The `system:completed_requests_history` keyspace is provided for {sqlpp} access to the archived files, but as they're external GZIP archives performance is restricted, particularly with large histories on clusters with multiple Query Service nodes.
911912
Directly reading the files may be more useful in some cases.
912913

913914
[#query-monitoring-settings]
@@ -922,7 +923,7 @@ You can set query profiling to the following levels:
922923

923924
You can set query profiling in the following ways:
924925

925-
* At the <<enable-settings-for-a-query-engine,node level>>, so that it is enabled for all queries on that node.
926+
* At the <<enable-settings-for-a-query-engine,node level>>, so that it's enabled for all queries on that node.
926927
* At the <<enable-settings-per-session-or-per-query,request level>>, for individual queries.
927928

928929
For more information about Query settings and parameters, see xref:n1ql:n1ql-manage/query-settings.adoc[].
@@ -1079,12 +1080,12 @@ When a query executes a user-defined function, profiling information is availabl
10791080

10801081
When profiling is enabled:
10811082

1082-
* If you are using the cbq shell or the Query REST API, query profiling information is returned with the query results.
1083-
* If you are using the Query workbench, query profiling information is not returned with the query results.
1083+
* If you're using the cbq shell or the Query REST API, query profiling information is returned with the query results.
1084+
* If you're using the Query workbench, query profiling information is not returned with the query results.
10841085

10851086
.Phases Profile
10861087
====
1087-
If you are using the cbq shell or the Query REST API, the following statistics are returned when `profile` is set to `phases`:
1088+
If you're using the cbq shell or the Query REST API, the following statistics are returned when `profile` is set to `phases`:
10881089
10891090
[source,json]
10901091
----
@@ -1142,7 +1143,7 @@ If you are using the cbq shell or the Query REST API, the following statistics a
11421143

11431144
.Timings Profile
11441145
====
1145-
If you are using the cbq shell or the Query REST API, the following statistics are returned when `profile` is set to `timings`:
1146+
If you're using the cbq shell or the Query REST API, the following statistics are returned when `profile` is set to `timings`:
11461147
11471148
[source,json]
11481149
----
@@ -1305,7 +1306,7 @@ For field names and meanings, see xref:n1ql-rest-query:index.adoc#Profile[Profil
13051306
[[plan]]
13061307
=== Profiling Details in System Catalogs
13071308

1308-
The <<sys-active-req,system:active_requests>> and <<sys-completed-req,system:completed_requests>> system catalogs always return profiling information regarding query phases: that is, phase times, phase counts, and phase operators.
1309+
The <<sys-active-req,system:active_requests>> and <<sys-completed-req,system:completed_requests>> system catalogs always return profiling information regarding query phases: namely, phase times, phase counts, and phase operators.
13091310

13101311
The <<sys-active-req,system:active_requests>>, <<sys-completed-req,system:completed_requests>>, and <<sys-prepared,system:prepareds>> system catalogs also support the `meta().plan` virtual attribute.
13111312
This captures the whole query plan, and includes profiling information regarding execution timings.
@@ -1507,4 +1508,4 @@ a| icon:check[fw] timings
15071508

15081509
== Related Links
15091510

1510-
* Refer to xref:n1ql:n1ql-intro/sysinfo.adoc[Getting System Information] for more information on the system namespace.
1511+
* For more information on the system namespace, see xref:n1ql:n1ql-intro/sysinfo.adoc[Getting System Information].

0 commit comments

Comments
 (0)