You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The program multiplies two matrices and displays the results on the console.
9
+
The program multiplies two matrices and prints the results on the screen.
10
10
The main purpose of the example is to show how to implement a user-defined CPU computing task.
11
11
12
12
# About computing tasks
13
13
14
-
A computing task has three arguments: INPUT, OUTPUT, and routine.
15
-
INPUT and OUTPUT are two template parameters that can hold any type, and a routine is the process from INPUT to OUTPUT. A computing task is defined as follows:
14
+
You need to provide three types of basic information when you define a computer task: INPUT, OUTPUT, and routine.
15
+
INPUT and OUTPUT are two template parameters, which can be of any type. routine means the process from INPUT to OUTPUT, which is defined as follows:
16
16
17
17
~~~cpp
18
18
template <classINPUT, class OUTPUT>
@@ -24,8 +24,8 @@ class __WFThreadTask
24
24
};
25
25
~~~
26
26
27
-
You can see that routine is a simple computing process from INPUT to OUTPUT. The INPUT pointer is not necessarily be const, but you can pass const INPUT \*.
28
-
For example, to implement a task that adds two numbers, you can use the follow code:
27
+
It can be seen that routine is a simple computing process from INPUT to OUTPUT. The INPUT pointer is not necessarily be const, but you can also pass the function of const INPUT \*.
28
+
For example, to implement an adding task, you can:
As the input matrices may be incompatible in matrix multiplication, there is an error field in the output to indicate that error.
79
+
As the input matrices may be illegal in matrix multiplication, so there is an error field in the output to indicate errors.
80
80
81
81
# Generating computing tasks
82
82
83
-
After you define the types of input and output and the algorithm process, you can use WFThreadTaskFactory to generate a computing task.
84
-
In [WFTaskFactory.h](../src/factory/WFTaskFactory.h), the calculation factory is defined as follows:
83
+
After you define the types of input and output and the algorithm process, you can use WFThreadTaskFactory to generate a computing task.
84
+
In [WFTaskFactory.h](../src/factory/WFTaskFactory.h), the computing task factory is defined as follows:
85
85
86
86
~~~cpp
87
87
template <classINPUT, class OUTPUT>
@@ -98,9 +98,9 @@ public:
98
98
};
99
99
~~~
100
100
101
-
Slightly different from the previous network factory class or the algorithm factory class, this class requires two template parameters: INPUT and OUTPUT.
102
-
queue\_name is explained in the previous example. routine is the computing process, and the callback means the callback of the function.
103
-
In the example, there is a call:
101
+
Slightly different from the previous network factory class or the algorithm factory class, this factory requires two template parameters: INPUT and OUTPUT.
102
+
queue\_name is explained in the previous example. routine is the computation process, and callback means the callback.
103
+
In our example, we see this call:
104
104
105
105
~~~cpp
106
106
using MMTask = WFThreadTask<algorithm::MMInput,
@@ -123,7 +123,7 @@ int main()
123
123
}
124
124
~~~
125
125
126
-
After the task is generated, you can use **get\_input()** interface to get the pointer of the input data. This is similar to the **get\_req()** in a network task.
126
+
After the task is generated, use **get\_input()** interface to get the pointer of the input data. This is similar to the **get\_req()** in a network task.
127
127
The start and the end of a task is the same as those of a network task. Similarly, the callback is very simple:
You can ignore the possibility of failure in ordinary computing tasks, and the end state is always SUCCESS.
151
+
You can ignore the the possibility of failure in the ordinary computing tasks, and the end state is always SUCCESS.
152
152
The callback simply prints out the input and the output. If the input data are illegal, the error will be printed out.
153
153
154
154
# Symmetry of the algorithm and the protocol
155
155
156
156
In our system, algorithms and protocols are highly symmetrical on a very abstract level.
157
-
There are thread tasks with user-defined algorithms, and obviously there are network tasks with user-defined protocols.
158
-
A user-defined algorithm requires a user to provide the procedure for algorithm implementation, and a user-defined protocol requires a user to provide the procedures for serialization and deserialization. You can see an introduction in [Simple client/server based on user-defined protocols](./tutorial-10-user_defined_protocol.md)
159
-
For user-defined algorithms and user-defined protocols, both algorithms and protocols must be very pure and have single responsibility.
160
-
For example, an algorithm is just a conversion procedure from INPUT to OUPUT, and the algorithm does not care about the existence of task, series, and etc.
161
-
The implementation of an HTTP protocol only cares about serialization and deserialization, and does not need to care about the task. Instead, an HTTP task refers to the HTTP protocol.
157
+
There are thread tasks with user-defined algorithms, obviously there are network tasks with user-defined protocols.
158
+
A user-defined algorithm requires the user to provide the algorithm procedure, and a user-defined protocol requires the user to provide the procedure of serialization and deserialization. You can see an introduction in [Simple client/server based on user-defined protocols](./tutorial-10-user_defined_protocol.md)
159
+
For the user-defined algorithms and the user-defined protocols, both must be very pure .
160
+
For example, an algorithm is just a conversion procedure from INPUT to OUPUT, and the algorithm does not know the existence of task, series, etc.
161
+
The implementation of an HTTP protocol only cares about serialization and deserialization, and does not need to care about the task definition. Instead, the HTTP protocol is referred to in an http task.
162
162
163
-
# Compositionality of thread tasks and network tasks
163
+
# Composite features of thread tasks and network tasks
164
164
165
165
In this example, we use WFThreadTaskFactory to build a thread task. This is the simplest way to get a computing task, and it is sufficient in most cases.
166
166
Similarly, you can simply define a server and a client with a user-defined protocol.
167
-
However, in the previous example, we use the algorithm factory to generate a parallel sorting task, which is obviously not possible with a routine.
167
+
However, in the previous example, we can use the algorithm factory to generate a parallel sorting task, which is obviously not possible with a routine.
168
168
For a network task, such as a Kafka task, interactions with several machines may be required to get results, but it is completely transparent to users.
169
-
Therefore, our tasks are composite. If you use our framework skillfully, you can design a lot of composite components.
169
+
Therefore, our tasks are composite. If you use our framework skillfully, you can design many composite components.
Copy file name to clipboardExpand all lines: docs/en/tutorial-09-http_file_server.md
+21-21Lines changed: 21 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,4 @@
1
-
# HTTP server with file IO: http\_file\_server
1
+
# Http server with file IO: http\_file\_server
2
2
3
3
# Sample code
4
4
@@ -7,12 +7,12 @@
7
7
# About http\_file\_server
8
8
9
9
http\_file\_server is a web server. You can start a web server after specifying the startup port and the root path (the default setting is the current path).
10
-
You can also start an HTTPS web server after specifying both a certificate file and a key file in PEM format.
11
-
This article mainly demonstrates how to use disk IO tasks. In the Linux system, we use the **aio** interface in the kernel of Linux, and the file reading is completely asynchronous.
10
+
You can also specify a certificate file and a key file in PEM format to start an HTTPS web server.
11
+
The program mainly demonstrates how to use disk IO tasks. In the Linux system, we use the aio interface in the kernel of Linux, and the file reading is completely asynchronous.
12
12
13
13
# Starting a server
14
14
15
-
Server startup procedure is almost the same as that of an echo server or an HTTP proxy, but here you can start an SSL server:
15
+
For starting a server, the steps are almost the same as those when starting an echo server or an HTTP proxy. There is one more way to start an SSL server here:
16
16
17
17
~~~cpp
18
18
classWFServerBase
@@ -23,7 +23,7 @@ class WFServerBase
23
23
};
24
24
~~~
25
25
26
-
In other words, you can specify a cert file and a key file in PEM format and then start an SSL server.
26
+
In other words, you can specify a cert file and a key file in PEM format to start an SSL server.
27
27
In addition, when you define a server, you can use **std::bind()** to bind a root parameter to the process. The root parameter means the root path of the service.
28
28
29
29
~~~cpp
@@ -47,7 +47,7 @@ int main(int argc, char *argv[])
47
47
# Handling requests
48
48
49
49
Similar to http\_proxy, no threads are occupied in file reading. Instead, an asynchronous task is generated to read files, and a reply to the request is generated after the reading is completed.
50
-
Please note again that the complete reply data should be read into the memory before the reply is sent. Therefore, it is not suitable for transferring large files.
50
+
Please note again that the complete reply data should be read into the memory before the reply message is sent. Therefore, it is not suitable for transferring very large files.
Unlike http\_proxy that generates a new HTTP client task, here a **pread** task is generated by the factory.
82
+
Unlike http\_proxy that generates a new HTTP client task, here a pread task is generated by the factory.
83
83
[WFAlgoTaskFactory.h](../src/factory/WFTaskFactory.h) contains the definitions of relevant interfaces.
84
84
85
85
~~~cpp
@@ -109,9 +109,9 @@ public:
109
109
};
110
110
~~~
111
111
112
-
Both **pread** and **pwrite** return WFFileIOTask. The same principle also applies to **sort** and **psort**, and to client tasks and server tasks.
113
-
In addition to these two interfaces, **preadv** and **pwritev** return WFFileVIOTask; **fsync** and **fdsync** return WFFileSyncTask. You can see the details in the header file.
114
-
Currently, our interface requires users to open and close **fd** by themselves. We are developing a set of file management interfaces. In the future, users only need to pass file names, which is more friendly to cross-platform applications.
112
+
Both pread and pwrite return WFFileIOTask. We do not distinguish between sort and psort, and we do not distinguish between client and server task. They all follow the same principle.
113
+
In addition to these two interfaces, preadv and pwritev return WFFileVIOTask; fsync and fdsync return WFFileSyncTask. You can see the details in the header file.
114
+
Currently, our interface requires users to open and close fd by themselves. We are developing a set of file management interfaces. In the future, users only need to pass file names, which is more friendly to cross-platform applications.
115
115
The example uses the user\_data field of the task to save the global data of the service. For larger services, we recommend to use series context. You can see the [proxy examples](../tutorial/tutorial-05-http_proxy.cc) for details.
# About the implementation of the asynchronous file IO
146
+
# About the implementation of the file IO
147
147
148
-
Linux operating system supports a set of asynchronous IO calls with high efficiency and very little CPU occupation. If you use our framework in a Linux system, this set of interfaces are used by default.
149
-
We have implemented a set of posix aio interfaces to support other UNIX systems, and use the **sigevent** notification method of threads, but it is no longer in use because of its low efficiency.
150
-
Currently, for non-Linux systems, asynchronous IO is always realized through multithreading. When an IO task arrives, a thread is created in real time to execute the IO task, and then a callback is used to return to the handler thread pool.
151
-
Multi-threaded IO is also the only choice in macOS, because macOS does not have good **sigevent** support and posix aio does not work in macOS.
152
-
Multithreaded IO does not support preadv and pwritev tasks. When these two tasks are created and run, you will get an ENOSYS error in the callback.
153
-
Some UNIX systems do not support fdatasync. In this case, an fdsync task is equivalent to an fsync task.
148
+
Linux operating system supports a set of asynchronous IO system calls with high efficiency and very little CPU occupation. If you use our framework in a Linux system, this set of interfaces are used by default.
149
+
We have implemented a set of posix aio interfaces to support other UNIX systems, and used the sigevent notification method of threads, but it is no longer in use because of its low efficiency.
150
+
Currently, for non-Linux systems, asynchronous IO is always simulated by multi-threading. When an IO task arrives, a thread is created in real time to execute IO tasks, and then a callback is used to return to the handler thread pool.
151
+
Multi-threaded IO is also the only choice in macOS, because macOS does not have good sigevent support and posix aio will not work in macOS.
152
+
Multi-threaded IO does not support preadv and pwritev tasks. When these two tasks are created and run, you will get an ENOSYS error in the callback.
153
+
Some UNIX systems do not support fdatasync. In this case, an fdsync task is equivalent to an fsync task.
0 commit comments