Release 2.0.1 (25th Jul 2019)

C-Producer - Updates
Rotating file logger (small feature).
Validating Stream description returned from Describe stream API call and warning if there is a mismatch.
Additional tests and minor compiler warning fixes.
CPP Producer - Updates
Memory leak fix.
Minor header include change.
Test stability fix for RPi.
Platform Independent Code - Updates
Enabling put frame calls during async stream creation.
Fixes for memory corruption on 32 bit platforms.
Using PTS for file upload cases where DTS and frame durations are 0.
Fix invalid error ACK timestamp mapping causing rollback/retry logic to issue a warning.
Improved test coverage. More stability for RPi tests.
Kinesis Video GStreamer Plugin version matching SDK version.

Author: chehefen

.gitignore

@@ -13,3 +13,4 @@ kinesis-video-native-build/test_jpeg.mkv
 test-stream/
 *.key
 *.gpg
+cm-file

README.md

@@ -61,13 +61,32 @@ Please follow the **installation instructions** applicable to your operating sys
 
   [Kinesis Video Producer SDK CPP documentation](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/producer-sdk-cpp.html)
 
+  [Kinesis Video C Producer documentation](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/producer-sdk-c-api.html)
+
 ### License
 
-This library is licensed under the Apache License, 2.0. 
+This library is licensed under the Apache License, 2.0.
 
 ----
 ### Release notes
 
+#### Release 2.0.1 (25th Jul 2019)
+* C-Producer - Updates
+  * Rotating file logger (small feature).
+  * Validating Stream description returned from Describe stream API call and warning if there is a mismatch.
+  * Additional tests and minor compiler warning fixes.
+* CPP Producer - Updates
+  * Memory leak fix.
+  * Minor header include change.
+  * Test stability fix for RPi.
+* Platform Independent Code - Updates
+  * Enabling put frame calls during async stream creation.
+  * Fixes for memory corruption on 32 bit platforms.
+  * Using PTS for file upload cases where DTS and frame durations are 0.
+  * Fix invalid error ACK timestamp mapping causing rollback/retry logic to issue a warning.
+  * Improved test coverage. More stability for RPi tests.
+* Kinesis Video GStreamer Plugin version matching SDK version.
+
 #### Release 2.0.0 (9th Jul 2019)
 * License update: KVS SDK and GStreamer kvssink plugin are under Apache 2.0 license now.
 * C-Producer - New feature
@@ -87,7 +106,7 @@ This library is licensed under the Apache License, 2.0.
 
 * Please note the following changes if you have customized the SDK for your integration and not using samples directly.
   * Changes to utility functions:
-    * RotatingStaticCredentialProvider is removed from C++ producer. Customer can use StaticCredentialProvider instead. 
+    * RotatingStaticCredentialProvider is removed from C++ producer. Customer can use StaticCredentialProvider instead.
     * CredentialProviderUtil and SerializedCredentials are removed from C++ producer.
   * Following functions are removed from DefaultCallbackProvider: sleepUntilWithTimeCallback, shutdownStream, notifyResult, getControlPlaneUri, safeFreeBuffer, getStreamStatusFromString.
   * Changes to the object model:

docker_native_scripts/raspberry-pi-docker/Dockerfile

@@ -6,13 +6,23 @@ RUN apt-get update && apt-get install -y \
 	pkg-config \
 	cmake \
 	vim \
-	binutils \
 	make \
 	openssh-server \
 	gcc \
 	g++ \
-	python \
-	bzip2
+	bzip2 \
+	libssl-dev \
+	libcurl4-openssl-dev \
+	liblog4cplus-dev \
+	libgstreamer1.0-dev \
+	libgstreamer-plugins-base1.0-dev \
+	gstreamer1.0-plugins-base-apps \
+	gstreamer1.0-plugins-bad \
+	gstreamer1.0-plugins-good \
+	gstreamer1.0-plugins-ugly \
+	gstreamer1.0-tools \
+	gstreamer1.0-omx
+
 
 WORKDIR /opt/
 RUN curl -OL https://github.com/raspberrypi/firmware/archive/1.20180417.tar.gz
@@ -36,9 +46,8 @@ RUN     ln -s /opt/vc/lib/libopenmaxil.so /usr/lib/libopenmaxil.so && \
 WORKDIR /opt/
 RUN git clone https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp
 WORKDIR /opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/
-RUN chmod a+x install-script
-RUN ./install-script -j 4 -d
+RUN chmod a+x min-install-script
+RUN ./min-install-script
 
 WORKDIR /opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/
-ENV PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/downloads/local/bin:$PATH
-ENV GST_PLUGIN_PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/downloads/local/lib:$GST_PLUGIN_PATH
+ENV GST_PLUGIN_PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/:$GST_PLUGIN_PATH

docker_native_scripts/raspberry-pi-docker/README.md

@@ -56,10 +56,6 @@ Follow the next few steps to start Producer SDK GStreamer plugin element for sen
 
 ##### 4.1 Set these environment variable:
 
-`export LD_LIBRARY_PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/downloads/local/lib:$LD_LIBRARY_PATH`
-
-`export PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/downloads/local/bin:$PATH`
-
 `export GST_PLUGIN_PATH=/opt/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/downloads/local/lib:$GST_PLUGIN_PATH`
 
 

install-instructions-linux.md

@@ -381,6 +381,21 @@ Refer sample configuration in the folder `kinesis-video-native-build` for detail
   By default, the log configuration `log4cplus.rootLogger=DEBUG, KvsConsoleAppender` creates console appender (KvsConsoleAppender) which outputs the log messages in the console.
 * Log output messages to file: By adding file appender (KvsFileAppender) in the _rootLogger_ of log4cplus as `log4cplus.rootLogger=DEBUG, KvsConsoleAppender, KvsFileAppender` the debug messages will be stored in `kvs.log` file in the sub-folder `log` within `kinesis-video-native-build` directory.  The filename for the logs and the location can be modified by changing the line ` log4cplus.appender.KvsFileAppender.File=./log/kvs.log`
 
+----
+#####  How to enable saving c producer log into files.
+
+By default C producer prints all logging information to stdout.
+
+To send log information to a file (named kvsProducerLog.index), you need to use the addFileLoggerPlatformCallbacksProvider API after ClientCallbacks has been initialized.
+
+The addFileLoggerPlatformCallbacksProvider API takes five parameters.
+
+* First parameter is the PClientCallbacks that is created during the createCallback provider API (e.g.createDefaultCallbacksProviderWithAuthCallbacks.
+* Second parameter is the size of string buffer that file logger will use. Logs are buffered in the string buffer and flushed into files when the buffer is full.
+* Third parameter is the maximum number of files that file logger will generate. When the limit is reached, oldest log file will be deleted before creating the new one.
+* Fourth parameter is the absolute directory path to store the log file.
+* Fifth parameter uses boolean true or false and is used to allow printing logs to both stdout and a file (useful in debugging).
+
 ----
 
 ### Open Source Dependencies

install-instructions-macos.md

@@ -219,13 +219,13 @@ AWS_ACCESS_KEY_ID=YourAccessKeyId AWS_SECRET_ACCESS_KEY=YourSecretAccessKey ./ki
 ##### Using Docker to run the sample application
 Refer the **README.md** file in the  *docker_native_scripts* folder for running the build and **RTSP sample application** to start streaming from **IP camera** within Docker container.
 
-### How to run sample applications for sending H264 video files to KVS 
- 
+### How to run sample applications for sending H264 video files to KVS
+
 The sample application `kinesis_video_cproducer_video_only_sample` sends h264  video frames inside the folder `kinesis-video-c-producer/samples/h264SampleFrames` to KVS.
-The following command sends the video frames in a loop for ten seconds to KVS. 
+The following command sends the video frames in a loop for ten seconds to KVS.
 you can run `./kinesis_video_cproducer_video_only_sample YourStreamName 10`
 `./kinesis_video_cproducer_video_only_sample YourStreamName 10`
- 
+
 If you want to send H264 files from another folder (`MyH264FramesFolder`) you can run the sample with the following arguments
 `./kinesis_video_cproducer_video_only_sample YourStreamName 10 MyH264FramesFolder`
 
@@ -240,17 +240,17 @@ For additional examples on using Kinesis Video Streams Java SDK and  Kinesis Vid
 
 ##### Running C++ Unit tests
 
-**Note:** Please set the [credentials](https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/install-instructions-macos.md#setting-credentials-in-environment-variables) before running the unit tests. 
+**Note:** Please set the [credentials](https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/install-instructions-macos.md#setting-credentials-in-environment-variables) before running the unit tests.
 
 The executable for **unit tests** will be built as `./producer_test` inside the `kinesis-video-native-build` directory. Launch it and it will run the unit test and kick off dummy frame streaming.
 
 ##### Running C unit tests
 
-The executable for **unit tests** will be built as `./cproducer_test` inside the `kinesis-video-native-build` directory. 
+The executable for **unit tests** will be built as `./cproducer_test` inside the `kinesis-video-native-build` directory.
 
 ##### Running PIC unit tests
 
-The executable for **unit tests** will be built as `./pic_test` inside the `kinesis-video-native-build` directory. 
+The executable for **unit tests** will be built as `./pic_test` inside the `kinesis-video-native-build` directory.
 
 
 ##### Enabling verbose logs
@@ -266,6 +266,22 @@ Refer sample configuration in the folder `kinesis-video-native-build` for detail
   By default, the log configuration `log4cplus.rootLogger=DEBUG, KvsConsoleAppender` creates console appender (KvsConsoleAppender) which outputs the log messages in the console.
 * Log output messages to file: By adding file appender (KvsFileAppender) in the _rootLogger_ of log4cplus as `log4cplus.rootLogger=DEBUG, KvsConsoleAppender, KvsFileAppender` the debug messages will be stored in `kvs.log` file in the sub-folder `log` within `kinesis-video-native-build` directory.  The filename for the logs and the location can be modified by changing the line `log4cplus.appender.KvsFileAppender.File=./log/kvs.log`
 
+----
+#####  How to enable saving c producer log into files.
+
+By default C producer prints all logging information to stdout.
+
+To send log information to a file (named kvsProducerLog.index), you need to use the addFileLoggerPlatformCallbacksProvider API after ClientCallbacks has been initialized.
+
+The addFileLoggerPlatformCallbacksProvider API takes five parameters.
+
+* First parameter is the PClientCallbacks that is created during the createCallback provider API (e.g.createDefaultCallbacksProviderWithAuthCallbacks.
+* Second parameter is the size of string buffer that file logger will use. Logs are buffered in the string buffer and flushed into files when the buffer is full.
+* Third parameter is the maximum number of files that file logger will generate. When the limit is reached, oldest log file will be deleted before creating the new one.
+* Fourth parameter is the absolute directory path to store the log file.
+* Fifth parameter uses boolean true or false and is used to allow printing logs to both stdout and a file (useful in debugging).
+
+
 ----
 #### Troubleshooting:
 

install-instructions-windows-msys2.md

@@ -183,3 +183,28 @@ gst-launch-1.0 -v ksvideosrc ! videoconvert ! x264enc bframes=0 key-int-max=45 b
 ```
 gst-launch-1.0 -v ksvideosrc ! h264parse ! video/x-h264,stream-format=avc,alignment=au ! kvssink name=sink stream-name=stream-name access-key=YOUR_ACCESS_KEY secret-key=YOUR_SECRET_ACCESS_KEY_ID wasapisrc device="\{0.0.1.00000000\}.\{f1245929-0c97-4389-9c28-1ca9cb01576b\}" ! audioconvert ! avenc_aac ! queue ! sink.
 ```
+
+----
+#####  How to configure logging for producer SDK sample applications.
+
+For the sample applications included in the producer SDK (CPP), the log configuration is referred from the file  `kvs_log_configuration` (within the `kinesis-video-native-build` folder).
+
+Refer sample configuration in the folder `kinesis-video-native-build` for details on how to set the log level (DEBUG or INFO) and output options (whether to send log output to either console or file (or both)).
+* Log output messages to console:
+  By default, the log configuration `log4cplus.rootLogger=DEBUG, KvsConsoleAppender` creates console appender (KvsConsoleAppender) which outputs the log messages in the console.
+* Log output messages to file: By adding file appender (KvsFileAppender) in the _rootLogger_ of log4cplus as `log4cplus.rootLogger=DEBUG, KvsConsoleAppender, KvsFileAppender` the debug messages will be stored in `kvs.log` file in the sub-folder `log` within `kinesis-video-native-build` directory.  The filename for the logs and the location can be modified by changing the line `log4cplus.appender.KvsFileAppender.File=./log/kvs.log`
+
+----
+#####  How to enable saving c producer log into files.
+
+By default C producer prints all logging information to stdout.
+
+To send log information to a file (named kvsProducerLog.index), you need to use the addFileLoggerPlatformCallbacksProvider API after ClientCallbacks has been initialized.
+
+The addFileLoggerPlatformCallbacksProvider API takes five parameters.
+
+* First parameter is the PClientCallbacks that is created during the createCallback provider API (e.g.createDefaultCallbacksProviderWithAuthCallbacks.
+* Second parameter is the size of string buffer that file logger will use. Logs are buffered in the string buffer and flushed into files when the buffer is full.
+* Third parameter is the maximum number of files that file logger will generate. When the limit is reached, oldest log file will be deleted before creating the new one.
+* Fourth parameter is the absolute directory path to store the log file.
+* Fifth parameter uses boolean true or false and is used to allow printing logs to both stdout and a file (useful in debugging).

kinesis-video-c-producer/CMakeLists.txt

@@ -159,6 +159,7 @@ else()
             ${KINESIS_VIDEO_PRODUCER_C_SRC}/src/source/StreamErrorReportStateMachine.c
             ${KINESIS_VIDEO_PRODUCER_C_SRC}/src/source/StreamLatencyStateMachine.c
             ${KINESIS_VIDEO_PRODUCER_C_SRC}/src/source/StreamCallbacksProvider.c
+            ${KINESIS_VIDEO_PRODUCER_C_SRC}/src/source/FileLoggerPlatformCallbackProvider.c
             ${KINESIS_VIDEO_PRODUCER_C_SRC}/src/source/Version.c)
 
     include_directories(${KINESIS_VIDEO_PIC_SRC}/src/client/include)

kinesis-video-c-producer/src/include/com/amazonaws/kinesis/video/cproducer/Include.h

@@ -50,6 +50,7 @@ extern "C" {
 #define STATUS_DUPLICATE_STREAM_CALLBACK_FREE_FUNC                                  STATUS_PRODUCER_BASE + 0x0000001a
 #define STATUS_DUPLICATE_AUTH_CALLBACK_FREE_FUNC                                    STATUS_PRODUCER_BASE + 0x0000001b
 #define STATUS_DUPLICATE_API_CALLBACK_FREE_FUNC                                     STATUS_PRODUCER_BASE + 0x0000001c
+#define STATUS_FILE_LOGGER_INDEX_FILE_TOO_LARGE                                     STATUS_PRODUCER_BASE + 0x0000001d
 
 
 /**
@@ -805,6 +806,20 @@ PUBLIC_API STATUS freeContinuousRetryStreamCallbacks(PStreamCallbacks*);
  */
 PUBLIC_API STATUS createAbstractDefaultCallbacksProvider(UINT32, BOOL, UINT64, PCHAR, PCHAR, PCHAR, PCHAR, PCHAR, PClientCallbacks*);
 
+/**
+ * Use file logger instead of default logger which log to stdout. The underlying objects are automatically freed
+ * when PClientCallbacks is freed.
+ *
+ * @param - PClientCallbacks - IN - The callback provider whose logPrintFn will be replaced with file logger log printing function
+ * @param - UINT64 - IN - Size of string buffer in file logger. When the string buffer is full the logger will flush everything into a new file
+ * @param - UINT64 - IN - Max number of log file. When exceeded, the oldest file will be deleted when new one is generated
+ * @param - PCHAR - IN - Directory in which the log file will be generated
+ * @param - BOOL - IN - print log to std out too
+ *
+ * @return - STATUS code of the execution
+ */
+PUBLIC_API STATUS addFileLoggerPlatformCallbacksProvider(PClientCallbacks, UINT64, UINT64, PCHAR, BOOL);
+
 #pragma pack(pop, include)
 
 #ifdef  __cplusplus

kinesis-video-c-producer/src/source/AwsV4Signer.c

@@ -94,7 +94,7 @@ STATUS signCurlRequest(PCurlRequest pCurlRequest)
                                    hmac, &hmacSize));
 
     // Increment the curSize to account for the NULL terminator that's required by the hex encoder
-    hexHmacLen = SIZEOF(hexHmac) / SIZEOF(CHAR);
+    hexHmacLen = ARRAY_SIZE(hexHmac);
     CHK_STATUS(hexEncodeCase(hmac, hmacSize, hexHmac, &hexHmacLen, FALSE));
 
     // set the auth header

kinesis-video-c-producer/src/source/ContinuousRetryStreamCallbacks.c

@@ -25,7 +25,7 @@ STATUS createContinuousRetryStreamCallbacks(PClientCallbacks pCallbacksProvider,
     pContinuousRetryStreamCallbacks->pCallbacksProvider = (PCallbacksProvider) pCallbacksProvider;
 
     // Create the mapping table
-    CHK_STATUS(hashTableCreate(&pContinuousRetryStreamCallbacks->pStreamMapping));
+    CHK_STATUS(hashTableCreateWithParams(STREAM_MAPPING_HASH_TABLE_BUCKET_COUNT, STREAM_MAPPING_HASH_TABLE_BUCKET_LENGTH, &pContinuousRetryStreamCallbacks->pStreamMapping));
 
     // Create the guard locks
     pContinuousRetryStreamCallbacks->mappingLock = pContinuousRetryStreamCallbacks->pCallbacksProvider->clientCallbacks.createMutexFn(