The architecture used (not in detail) is shown in the figure above.
The RuSIEM kernel, web interface, backend and related components are fully developed by our team. In development are used:
С++ 11 (stl, boost), c# (.net
RuSIEM uses the following components and databases:
The product consists of:
Agent for Windows is used as an intermediary for native transports to collect from various sources other than syslog. One agent can collect both locally and simultaneously remotely from multiple sources using an agentless method. The communication channel between the agent and the server is encrypted. The agent has a built-in database for temporary storage of events when there is no connection to the server. The agent can work only with RuSIEM / RvSIEM, it is not compatible with other products.
Syslog sources are sent directly to the server.
The kernel (microservices) is more than 80% written in C ++. The analytics module, mainly java.
MQ (message queue) are used for communication of microservices, exchange between microservices of data pools, to ensure the transfer of events without loss in case of event peaks, restart of any component. The use of MQ is due to the fact that the loss of even one event can affect the detection of a threat, failure and incident.
Within the framework of one server (node), transfer between microservices is mainly carried out via the tcp protocol, but with the producer RuSIEM MQ, which controls the queues on all microservices. If it is impossible to process events on microservices (or peak node performance), the input queue is buffered in RuSIEM MQ directly to disk.
RuSIEM / RvSIEM nodes can be connected to each other using various methods:
Both fully complete nodes with a set of all microservices and databases can be scaled, as well as individual microservices. For example, on one node normalization, on the other correlation, on the third — the database of events.
It is allowed to connect a set of nodes of freely distributed RvSIEM to the nodes of a commercial version of RuSIEM.
Events are stored in RuSIEM / RvSIEM versions in the Elasticsearch database in json format. In this case, by default, both the raw event and the normalized event are saved. Each json contains both a raw event and a normalized one. Saving raw events can be disabled. Only normalized events are transmitted to the analytics module, raw events are not transmitted. The analytics module has its own set of databases and storage formats.
Database clusters scale very flexibly. Since database nodes are not limited to a license, you can build, for example: