-
Notifications
You must be signed in to change notification settings - Fork 724
Core Utilities
- Introduction
- Logging System
- Parameter Class
- Exception Hierarchy
- Arithmetic Utilities
- OS Abstraction Layer
- Base64 and MD5 Utilities
- TimerManager and SpendTimer
- LRUCache11 and ResourcePool
- Practical Examples
- Conclusion
This document provides comprehensive API documentation for the core utility classes in the hikyuu financial analysis framework. The utilities cover essential functionality for logging, configuration management, error handling, performance measurement, data encoding, and resource management. These components are designed to work seamlessly in both C++ and Python contexts, providing a consistent interface for developers. The documentation details each utility class, its methods, and practical usage patterns, enabling developers to effectively utilize these tools in their financial applications.
The logging system in hikyuu is built on the SPDLOG library, providing a flexible and efficient mechanism for recording application events at various severity levels. The system supports both file and console output with configurable formatting and color coding. The Log class provides a comprehensive set of macros and functions for logging messages with different severity levels, making it easy to implement detailed logging throughout the application.
The logging system defines six severity levels through the LOG_LEVEL enum:
- LOG_TRACE: Detailed tracing information for debugging
- LOG_DEBUG: Debug-level messages for development
- LOG_INFO: General information about application operation
- LOG_WARN: Warning messages indicating potential issues
- LOG_ERROR: Error messages indicating recoverable failures
- LOG_FATAL: Critical error messages indicating unrecoverable failures
The system can be initialized using the initLogger function, which configures the logger with optional parameters for color output and custom log file names. The default log file is "./hikyuu.log" in the current directory. The logging system also provides assertion macros like HKU_ASSERT and HKU_CHECK that automatically generate log entries when conditions fail, including stack trace information when enabled.
classDiagram
class Log {
+LOG_TRACE(...)
+LOG_DEBUG(...)
+LOG_INFO(...)
+LOG_WARN(...)
+LOG_ERROR(...)
+LOG_FATAL(...)
+initLogger(not_use_color, filename)
+get_log_level()
+set_log_level(level)
}
class LOG_LEVEL {
+LOG_TRACE
+LOG_DEBUG
+LOG_INFO
+LOG_WARN
+LOG_ERROR
+LOG_FATAL
+LOG_OFF
}
Log --> LOG_LEVEL : uses
Diagram sources
- Log.h
- Log.cpp
Section sources
- Log.h
- Log.cpp
The Parameter class provides a flexible configuration system that allows objects to have named parameters with various data types. This utility enables dynamic configuration of components without requiring changes to their class definitions. The class uses a map-based storage system with boost::any to support multiple data types, including int, bool, double, string, Stock, Block, KQuery, KData, PriceList, and DatetimeList.
The Parameter class supports several key operations:
-
Adding parameters: Using the
addmethod to create new parameters with default values -
Setting parameters: Using the
setmethod to modify existing parameter values -
Getting parameters: Using the
getmethod to retrieve parameter values with type safety -
Checking parameters: Using the
havemethod to verify if a parameter exists -
Listing parameters: Using
getNameListandgetNameValueListto retrieve parameter information
The class also provides template methods for type-safe parameter access and supports comparison operations for parameter state management. In Python, the Parameter class is accessible through the hikyuu module, allowing for seamless configuration management in both C++ and Python contexts.
classDiagram
class Parameter {
+support(value)
+getNameList()
+getNameValueList()
+have(name)
+size()
+type(name)
+set(name, value)
+get(name)
+tryGet(name, default)
}
class PARAMETER_SUPPORT {
+getParameter()
+setParameter(param)
+haveParam(name)
+setParam(name, value)
+getParam(name)
+tryGetParam(name, default)
}
Parameter <|-- PARAMETER_SUPPORT : implements
Diagram sources
- Parameter.h
- Parameter.cpp
Section sources
- Parameter.h
- Parameter.cpp
The exception hierarchy in hikyuu provides a robust error handling system based on the standard C++ exception mechanism. The core exception class is hku::exception, which inherits from std::exception and provides enhanced error reporting capabilities. This exception class supports both character string and std::string inputs, making it flexible for various error message formats.
The exception system is integrated with the logging system through assertion macros that automatically throw exceptions when conditions fail. These macros include:
- HKU_ASSERT: Throws an exception when a condition is false, with stack trace support
- HKU_CHECK: Throws an hku::exception when a condition is false
- HKU_CHECK_THROW: Throws a specified exception type when a condition is false
- HKU_THROW: Throws an hku::exception with a formatted message
- HKU_THROW_EXCEPTION: Throws a specified exception type with a formatted message
The system also provides conditional logging and return macros that combine logging with error handling, such as HKU_ERROR_IF_RETURN which logs an error message and returns a specified value when a condition is met. This integrated approach to error handling and logging ensures that errors are properly recorded and handled consistently throughout the application.
classDiagram
class exception {
+exception()
+exception(const char* msg)
+exception(const std : : string& msg)
+~exception()
+what()
}
class std : : exception {
+what()
}
exception --|> std : : exception : inherits
class AssertionMacros {
+HKU_ASSERT(expr)
+HKU_CHECK(expr, ...)
+HKU_CHECK_THROW(expr, except, ...)
+HKU_THROW(...)
+HKU_THROW_EXCEPTION(except, ...)
}
AssertionMacros ..> exception : uses
Diagram sources
- exception.h
- exception.cpp
- Log.h
Section sources
- exception.h
- exception.cpp
The arithmetic utilities in hikyuu provide a collection of mathematical and string manipulation functions specifically designed for financial calculations. These utilities include rounding functions with different modes (roundEx, roundUp, roundDown), string processing functions (to_lower, to_upper, trim, split), and specialized financial calculation helpers.
Key features of the arithmetic utilities include:
-
Rounding functions: The
roundExfunction implements traditional rounding (not banker's rounding) with configurable decimal places, whileroundUpandroundDownprovide ceiling and floor operations respectively - String utilities: Functions for case conversion, whitespace trimming, and string splitting that handle UTF-8 encoding properly across different platforms
-
Financial helpers: The
get_quantilefunction calculates quantiles from sorted vectors, useful for statistical analysis in financial applications - Encoding conversion: Functions to convert between UTF-8 and GB2312 encodings, important for handling Chinese characters in financial data on Windows platforms
The utilities also include functions for converting byte arrays to hexadecimal strings, which is useful for debugging and data representation. These functions are optimized for performance and handle edge cases appropriately, making them reliable for production use in financial applications.
classDiagram
class Arithmetic {
+roundEx(number, ndigits)
+roundUp(number, ndigits)
+roundDown(number, ndigits)
+to_lower(s)
+to_upper(s)
+trim(s)
+split(str, c)
+byteToHexStr(bytes, in_len)
+isInteger(num)
+get_quantile(vec, quantile)
}
class Encoding {
+utf8_to_gb(szinput)
+gb_to_utf8(szinput)
+HKU_STR(s)
+HKU_PATH(s)
}
Arithmetic ..> Encoding : includes
Diagram sources
- arithmetic.h
- arithmetic.cpp
Section sources
- arithmetic.h
- arithmetic.cpp
The OS abstraction layer in hikyuu provides a consistent interface for file system operations and system information retrieval across different operating systems. This utility layer encapsulates platform-specific details, allowing the rest of the codebase to work uniformly on Windows, Linux, macOS, and other supported platforms.
The os.h header defines a comprehensive set of functions for:
-
File operations: Checking file existence (
existFile), creating directories (createDir), removing files and directories (removeFile,removeDir), copying files (copyFile), and renaming files (renameFile) -
Path operations: Retrieving user directory (
getUserDir), current working directory (getCurrentDir), and DLL self directory (getDllSelfDir) -
System information: Getting platform name (
getPlatform), CPU architecture (getCpuArch), system language (getSystemLanguage), and memory information (getMemoryMaxSize,getMemoryIdleSize) -
Disk operations: Retrieving free disk space (
getDiskFreeSpace)
The implementation handles platform-specific differences in file system APIs, path separators, and encoding. For example, it automatically converts UTF-8 paths to GB2312 on Windows when necessary. The abstraction layer also provides a function to detect if the output terminal supports color codes (isColorTerminal), enabling conditional colored output in logs and console applications.
classDiagram
class OS {
+existFile(filename)
+createDir(pathname)
+removeFile(filename)
+removeDir(path)
+copyFile(src, dst, flush)
+renameFile(oldname, newname, overlay)
+getUserDir()
+getCurrentDir()
+getDllSelfDir()
+isColorTerminal()
+getDiskFreeSpace(path)
+getPlatform()
+getCpuArch()
+getSystemLanguage()
+getMemoryMaxSize()
+getMemoryIdleSize()
}
Diagram sources
- os.h
- os.cpp
Section sources
- os.h
- os.cpp
The base64 and MD5 utilities in hikyuu provide essential functions for data encoding, decoding, and integrity verification. These utilities are crucial for secure data handling, authentication, and data transmission in financial applications.
The base64 utility provides functions for:
-
Encoding: Converting binary data to base64 strings using
base64_encodewith support for URL-safe encoding -
Decoding: Converting base64 strings back to binary data using
base64_decode - Formatted encoding: Specialized functions for PEM and MIME formatted base64 encoding
The MD5 utility provides a reliable implementation of the MD5 hash algorithm for data integrity verification:
-
Hash calculation: The
md5function computes MD5 hashes from byte arrays or strings - Consistent results: The implementation ensures consistent hash results across different platforms, addressing issues with boost's MD5 implementation
- String convenience: Overloaded functions to directly hash std::string objects
These utilities are designed to be efficient and reliable, with proper error handling for invalid inputs. The base64 implementation handles edge cases like line breaks and padding correctly, while the MD5 implementation follows the standard algorithm specification to ensure compatibility with other systems.
classDiagram
class Base64 {
+base64_encode(bytes_to_encode, in_len, url)
+base64_encode(s, url)
+base64_encode_pem(s)
+base64_encode_mime(s)
+base64_decode(encoded_string, remove_linebreaks)
}
class MD5 {
+md5(input, len)
+md5(src)
}
class DataEncoding {
+Base64
+MD5
}
DataEncoding --> Base64
DataEncoding --> MD5
Diagram sources
- base64.h
- base64.cpp
- md5.h
- md5.cpp
Section sources
- base64.h
- base64.cpp
- md5.h
- md5.cpp
The TimerManager and SpendTimer utilities provide comprehensive timing and performance measurement capabilities for the hikyuu framework. These tools enable both scheduled task execution and detailed performance profiling.
The TimerManager class implements a sophisticated scheduling system that can:
-
Schedule recurring tasks: Execute functions at regular intervals with
addDurationFunc -
Schedule delayed tasks: Execute functions after a specified delay with
addDelayFunc -
Schedule at specific times: Execute functions at precise moments with
addFuncAtTime -
Schedule daily tasks: Execute functions at the same time every day with
addFuncAtTimeEveryDay - Manage task lifecycle: Add, remove, and monitor scheduled tasks
The TimerManager uses a priority queue to efficiently manage task execution order and supports multi-threaded execution through a configurable thread pool. It handles edge cases like system time adjustments and provides precise timing control for financial applications where timing accuracy is critical.
The SpendTimer utility provides performance profiling capabilities through a set of convenient macros:
- SPEND_TIME: Measure execution time of code blocks
- SPEND_TIME_MSG: Measure execution time with custom messages
- SPEND_TIME_KEEP: Record intermediate timings within a code block
- BENCHMARK_TIME: Measure performance of repeated operations
These tools work together to provide comprehensive timing capabilities, from high-level task scheduling to low-level performance measurement, enabling developers to optimize their financial algorithms and ensure timely execution of critical operations.
classDiagram
class TimerManager {
+TimerManager(work_num)
+TimerManager(tg)
+~TimerManager()
+start()
+stop()
+size()
+empty()
+stopped()
+addFunc(start_date, end_date, start_time, end_time, repeat_num, duration, f, args)
+addDurationFunc(repeat_num, duration, f, args)
+addDelayFunc(delay, f, args)
+addFuncAtTime(time_point, f, args)
+addFuncAtTimeEveryDay(start_date, end_date, time, f, args)
+addFuncAtTimeEveryDay(time, f, args)
+removeTimer(timerid)
}
class SpendTimer {
+SpendTimer()
+SpendTimer(id, filename, lineno)
+SpendTimer(id, msg, filename, lineno)
+~SpendTimer()
+duration()
+show()
+value()
+keep(description)
+getKeepDurations()
+setCycle(cycle)
+isClosed()
}
class TimingMacros {
+SPEND_TIME(id)
+SPEND_TIME_MSG(id, ...)
+SPEND_TIME_KEEP(id, ...)
+SPEND_TIME_SHOW(id)
+SPEND_TIME_VALUE(id)
+BENCHMARK_TIME(id, cycle)
+BENCHMARK_TIME_MSG(id, cycle, ...)
+SPEND_TIME_CONTROL(open)
+OPEN_SPEND_TIME
+CLOSE_SPEND_TIME
}
TimingMacros ..> SpendTimer : uses
Diagram sources
- TimerManager.h
- SpendTimer.h
Section sources
- TimerManager.h
- SpendTimer.h
The LRUCache11 and ResourcePool utilities provide efficient resource management capabilities for the hikyuu framework. These tools help optimize performance by caching frequently accessed data and managing expensive resources like database connections.
The LRUCache11 class implements a high-performance LRU (Least Recently Used) cache with O(1) operations for insert, delete, and refresh. Key features include:
- Template-based design: Supports any key and value types
- Configurable size: Limits cache size with optional elasticity
- Thread safety: Optional synchronization using custom lock types
- Efficient operations: Uses a combination of hash map and linked list for optimal performance
The ResourcePool class provides a generic resource pooling mechanism that can manage any type of resource, such as database connections or network sockets. It features:
- Parameter-based configuration: Resources are created with configurable parameters
- Size limits: Configurable maximum pool and idle sizes
- Thread-safe operations: Uses mutexes and condition variables for safe concurrent access
- Automatic cleanup: Resources are properly destroyed when the pool is destroyed
The ResourceVersionPool extends the basic ResourcePool with versioning capabilities, allowing for seamless parameter changes without disrupting ongoing operations. When parameters change, new resources use the updated parameters while existing resources continue with their original configuration until they are returned to the pool.
classDiagram
class LRUCache11 {
+Cache(maxSize, elasticity)
+~Cache()
+size()
+empty()
+clear()
+insert(k, v)
+tryGet(k, vOut)
+get(k)
+getCopy(k)
+remove(k)
+contains(k)
+getMaxSize()
+getElasticity()
+getMaxAllowedSize()
+cwalk(f)
}
class ResourcePool {
+ResourcePool(param, maxPoolSize, maxIdleNum)
+~ResourcePool()
+maxPoolSize()
+maxIdleSize()
+maxPoolSize(num)
+maxIdleSize(num)
+get()
+getWaitFor(ms_timeout)
+getAndWait()
+count()
+idleCount()
+releaseIdleResource()
}
class ResourceVersionPool {
+ResourceVersionPool(param, maxPoolSize, maxIdleNum)
+~ResourceVersionPool()
+haveParam(name)
+getParam(name)
+setParam(name, value)
+setParameter(param)
+setParameter(param)
+getVersion()
+incVersion(version)
+get()
+getWaitFor(ms_timeout)
+getAndWait()
+count()
+idleCount()
+releaseIdleResource()
}
ResourceVersionPool --|> ResourcePool : inherits
Diagram sources
- LRUCache11.h
- ResourcePool.h
Section sources
- LRUCache11.h
- ResourcePool.h
This section provides practical examples demonstrating the use of hikyuu's core utilities in both C++ and Python contexts.
In C++, a class can use the PARAMETER_SUPPORT macro to enable parameter configuration:
class MyStrategy {
PARAMETER_SUPPORT
public:
MyStrategy() {
addParam<int>("period", 20);
addParam<double>("threshold", 0.5);
}
void calculate() {
int period = getParam<int>("period");
double threshold = getParam<double>("threshold");
// Use parameters in calculation
}
};In Python, the same functionality is available:
param = Parameter()
param.add("period", 20)
param.add("threshold", 0.5)
strategy.setParameter(param)The integrated logging and exception system enables robust error handling:
void processData(const Data& data) {
HKU_CHECK(!data.empty(), "Data cannot be empty");
HKU_CHECK(data.size() <= MAX_SIZE, "Data size {} exceeds limit {}",
data.size(), MAX_SIZE);
try {
// Process data
} catch (const std::exception& e) {
HKU_ERROR("Processing failed: {}", e.what());
HKU_THROW("Failed to process data");
}
}Using SpendTimer to measure performance:
void criticalFunction() {
SPEND_TIME(algorithm, "Executing trading algorithm");
// Algorithm steps
SPEND_TIME_KEEP(algorithm, "Loaded market data");
// More processing
SPEND_TIME_KEEP(algorithm, "Calculated indicators");
// Final steps
SPEND_TIME_SHOW(algorithm); // Print total time
}Using base64 and MD5 for data integrity:
std::string encodeAndVerify(const std::string& data) {
// Encode data
std::string encoded = base64_encode(data);
// Verify integrity
std::string hash = md5(data);
return encoded + ":" + hash;
}Section sources
- Parameter.h
- Log.h
- SpendTimer.h
- base64.h
- md5.h
The core utilities in hikyuu provide a comprehensive foundation for building robust financial applications. The logging system offers detailed insight into application behavior with configurable severity levels and output formats. The Parameter class enables flexible configuration management that works seamlessly across C++ and Python. The exception hierarchy ensures consistent error handling throughout the codebase.
The arithmetic utilities provide essential mathematical and string functions optimized for financial calculations, while the OS abstraction layer ensures cross-platform compatibility for file system operations. The base64 and MD5 utilities enable secure data handling and integrity verification. The TimerManager and SpendTimer classes offer comprehensive timing and performance measurement capabilities, from scheduled task execution to detailed code profiling.
Finally, the LRUCache11 and ResourcePool classes provide efficient resource management, helping to optimize performance by caching frequently accessed data and managing expensive resources. Together, these utilities form a powerful toolkit that enables developers to build high-performance, reliable financial applications with proper logging, configuration, error handling, and performance monitoring.