You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TigerGraph Database captures key information on activities occurring across its different components through log functions that output to log files. These log files are not only helpful in xref:troubleshooting-guide.adoc[troubleshooting] but also serve as an auditory resource. This document gives a high-level overview of TigerGraph's logging structure and lists some common information one might need to monitor their database services and where to obtain them in the logs.
4
4
5
-
== Overall Logging Structure
5
+
== Available Log Files
6
6
7
-
Logs in TigerGraph are stored at `<tigergraph_root_dir>/log/`. TigerGraph's logs are divided into different folders by the different internal components and each folder corresponds to a different component. Log formats also vary across the different components. In folders where logs are checked often, such as `restpp`, `gsql`, and `admin`, there are three symbolic links that help you quickly get to the most recent log file of that category:
7
+
TigerGraph generates a variety of log files for its different components.
8
+
Understanding what logs are available and what they contain is the first step in effective troubleshooting and system monitoring.
8
9
9
-
* `log.INFO`
10
-
** Contains regular output and errors
11
-
* `log.ERROR`
12
-
** Contains errors only
13
-
* `<component_name>.out`
14
-
** Contains all output from the component process
15
-
* `log.WARNING` or `log.DEBUG`
16
-
** `log.WARNING` contains warnings
17
-
** In the``gsql`` folder, `log.DEBUG` contains very specific information you only need when certain errors happen
10
+
=== Log File Locations
11
+
12
+
Logs in TigerGraph are stored in the log root directory, which is configured at install time. You can find this location by running:
13
+
14
+
[source,console]
15
+
----
16
+
gadmin config get System.LogRoot
17
+
----
18
+
19
+
Within this directory, you will find subdirectories for each TigerGraph component (admin, gpe, gsql, gui, kafka, nginx, zk, etc.).
Knowing where certain activities are recorded allows one to use tools such as the Linux `grep` command to easily obtain critical information from your database.
51
+
Third-Party components like Zookeeper and Kafka have logs that are not listed by `gadmin log`. You can find them at:
20
52
21
-
== Log locations on a cluster
53
+
[source,console]
54
+
----
55
+
zookeeper : ~/tigergraph/zk/zookeeper.out.*
56
+
kafka : ~/tigergraph/kafka/kafka.out
57
+
----
22
58
23
-
In a TigerGraph cluster, each node will only keep logs of activities that took place on the node itself. For example, the GSQL logs on the m1 node will only record events for m1 and are not replicated across the cluster.
59
+
In a *TigerGraph cluster*, each node maintains logs only for the activities that occur on that node. Logs are not automatically replicated across nodes.
60
+
For example, the GSQL logs on the m1 node reflect only the operations performed on m1.
61
+
To determine which node is currently the GSQL leader, check the most recent `log.INFO` file on m1.
24
62
25
-
For GSQL specifically, the cluster will elect a leader to which all GSQL requests will be forwarded. To check which node is the leader, start by checking the GSQL logs of the m1 node. Check the most recent lines of log.INFO and look for lines containing information about leader switch. For example, the logs below recorded a GSQL leader switch from m2 to m1:
63
+
For example, the logs below recorded a GSQL leader switch from m2 to m1:
26
64
27
65
[,console]
28
66
----
@@ -34,6 +72,81 @@ I@20210709 13:56:52.220 (SessionManager.java:204) All sessions aborted.
34
72
I@20210709 13:56:52.224 (GsqlHAHandler.java:283) switched to new leader m1
35
73
----
36
74
75
+
=== TigerGraph Component Log Files
76
+
77
+
* `.out` files capture *standard output (stdout)* and log runtime information, including error stack traces when services crash or unexpected errors occur.
78
+
These logs are especially useful for errors that aren't logged by the service's internal logging mechanism.
79
+
80
+
* `.ERROR` files are used to log errors captured by the system, typically from exceptions caught in try-catch blocks. If an error occurs before the logging system initializes or is uncaught, it is logged in the `.out` file instead.
81
+
82
+
* `.INFO` files log regular operational information about the system's normal functioning.
83
+
84
+
To diagnose an issue for a given component, check the `.out` log file for that component.
The GUI component writes all log levels to a single log file and does not generate separate `.log`, `.error`, or `.info` files.
91
+
92
+
* Each GUI log file (e.g., `gui_ADMIN.log`, `gui_INFO.log`) captures the standard output of the GUI process and includes all log levels (error, warning, info).
93
+
* The log level for the GUI component is *configurable*. You can set it using:
94
+
95
+
[source,console]
96
+
----
97
+
gadmin config set GUI.BasicConfig.LogConfig.LogLevel <LEVEL>
98
+
----
99
+
100
+
Replace `<LEVEL>` with one of: `DEBUG`, `INFO`, `WARN`, `ERROR`, `PANIC`, or `FATAL`. The default level is `INFO`.
101
+
====
102
+
103
+
==== Symbolic Links
104
+
105
+
In directories with frequently checked logs, such as `restpp`, `gsql`, and `admin`, symbolic links make it easier to access the latest log file.
106
+
These links are automatically updated to point to the newest log.
107
+
108
+
For example, `log.INFO` is a symbolic link that points to the current `.INFO` log file. To see what a symbolic link points to, use `ls -ll` followed by the symbolic link name:
109
+
110
+
[source,console]
111
+
----
112
+
ls -ll log.INFO
113
+
log.INFO -> log.INFO.2024-07-01-10-00-00
114
+
----
115
+
116
+
Here, `log.INFO` is a symbolic link pointing to the current `.INFO` log file.
117
+
118
+
=== Third-Party Component Log Files
119
+
120
+
TigerGraph uses several open-source components (such as Kafka, Nginx, ZooKeeper, Kafkaconn, Kafkastream) that maintain their own log conventions.
121
+
122
+
* *NGINX Logs:* The NGINX log files (e.g., `nginx.out`, `nginx.error.log`, `nginx.access.log`) are generated directly by the NGINX web server itself and are not internal TigerGraph component logs.
123
+
124
+
* *Kafka Logs:* Kafka logs include `controller.log`, `kafka.log`, `kafka-request.log`, `state-change.log`, and `server.log`.
125
+
126
+
* *ZooKeeper Logs:* ZooKeeper logs are typically found as `zookeeper.out.*` in the ZooKeeper directory.
127
+
128
+
== TigerGraph log structure
129
+
130
+
[CAUTION]
131
+
====
132
+
Log formats may differ between `.out` and `.INFO` logs and between different TigerGraph services.
133
+
====
134
+
135
+
* `log.INFO`: Contains regular output and errors.
136
+
* `log.ERROR`: Contains errors only.
137
+
* `<component_name>.out`: Contains all output from the component process. Current `.out` logs have the form `<service name>.out`. Historical logs have the form `<service name>-old-YYYY-MM-DDTHH-MM-SS.fff.out`
138
+
* `log.WARNING` or `log.DEBUG`
139
+
** `log.WARNING` contains warnings and all error-level messages.
140
+
** `log.DEBUG` contains debug-level messages (not created by default).
141
+
* `log.FATAL`: Contains outputs for any fatal level events
142
+
143
+
[NOTE]
144
+
====
145
+
All services do not create a `log.DEBUG` file by default.
146
+
To change this, modify the parameter `<service>.BasicConfig.LogConfig.LogLevel`.
147
+
For example, `GSQL.BasicConfig.LogConfig.LogLevel`. See xref:reference:configuration-parameters.adoc[Configuration Parameters] for more information.
148
+
====
149
+
37
150
== Monitor request history
38
151
39
152
All requests made to TigerGraph's REST endpoints are recorded by the RESTPP logs and Nginx logs. Information available in the logs includes:
0 commit comments