What is Elastic Stack?
Elastic Stack, also known as ELK (Elasticsearch + Logstash + Kibana), is probably the most widely known and most used system for collecting and analyzing logs, metrics, and other data about the state of systems – servers, clusters, clouds.
It consists of three main components:
In addition, there is a set of so-called Beats for ELK (I’ll call it that out of habit), which are data collection utilities. Among them, for example, Filebeat – for collecting data from files (logs), or Metricbeat – for collecting data about the system – processor, RAM, etc.
The stack operation looks like this:
What is Elasticsearch?
Elasticsearch is a distributed search and analysis system based on Apache Lucene. Shortly after its release in 2010, Elasticsearch became the most popular search engine and is commonly used for applications such as log analysis, full-text search, intelligent security, business intelligence, and process monitoring.
On January 21, 2021, Elastic NV announced a change in its software licensing strategy and that new versions of Elasticsearch and Kibana will not be released under the Apache License version 2.0 (ALv2) permissive license. Instead, new versions of the software are offered under the Elastic license, and the source code is available under the Elastic or SSPL license. These licenses are not open source and do not give users the same freedom. In an effort to provide open source professionals and our customers with a secure, high quality, fully open source search and analytics toolkit, we created the OpenSearch project, a community-developed offshoot of Elasticsearch and Kibana that is open source and licensed under the ALv2 license.
How does Elasticsearch work?
You can send data to Elasticsearch as JSON documents using APIs or ingestion tools such as Logstash and Amazon Kinesis Firehose. Elasticsearch automatically saves the original document and adds a link to it to the cluster index, including search capabilities. You can then search and retrieve the document using the Elasticsearch API. You can also use Kibana, a visualization tool with Elasticsearch, to visualize data and create interactive dashboards.
Versions of Elasticsearch licensed under the Apache 2.0 license (prior to 7.10.2 and Kibana 7.10.2) can be run on-premises, on Amazon EC2, or on Amazon OpenSearch. If you are deploying on-premises or on Amazon EC2, you are responsible for installing Elasticsearch and other required software, preparing the infrastructure, and managing the cluster. Amazon OpenSearch, on the other hand, is a fully managed service, so you don’t have to worry about the time-consuming process of managing clusters and tasks such as hardware preparation, software patching, disaster recovery, backup, and monitoring.
Advantages of Elasticsearch
Favorable price-to-performance ratio
Elasticsearch offers simple REST-based APIs and a lightweight HTTP interface, and uses schema-free JSON documents to get started and quickly build applications for a variety of use cases.
High performance
Elasticsearch’s distributed system allows you to process large amounts of data in parallel, instantly finding the best match for your query.
Free tools and modules
Elasticsearch is built into Kibana, a popular visualization and reporting tool. Integration with Beats and Logstash is also available, and source data can be easily converted and uploaded to the Elasticsearch cluster. You can use a number of open source Elasticsearch plugins, such as language analyzers and recommendation engines, to extend the functionality of your applications.
Real-time operations
Operations in Elasticsearch, such as reading or writing data, typically take less than a second. This allows you to use it in applications where you need to react in near real-time, such as application monitoring and anomaly detection.
Easy application development
Elasticsearch provides support for a variety of languages, including Java, Python, PHP, JavaScript, Node.js, Ruby, and many others.
What is Elastic Observability?
Transforming data to a standardized view – converging metrics, logs, and traces to provide a unified view.
What is Elastic Security
The Elastic Security platform enables analysts to prevent, detect, and respond to threats. Elastic Security supports Elastic Common Schema (ECS), a new specification that provides a consistent and customizable way to structure data in Elasticsearch, making it easier to analyze data from multiple sources. With ECS, you can use analytical content such as dashboards and machine learning tasks more broadly, search queries are narrower, and field names are easier to remember.
Elastic Security has a threat detection engine that automatically detects threats, reducing detection time and giving security teams more time to perform tasks that require expert input.
Elastic Security analyzes any data, automates key processes, and protects the OS. It is unified security on an open platform: