Elastic

Home  ›  Software  ›  Elastic
Elastic

Elastic Stack – An extensive ecosystem of components that serve to search and process data

 

 

Elastic Global Threat Report for 2023

What is Elastic Stack?

Elastic Stack, also known as ELK (Elasticsearch + Logstash + Kibana), is probably the most widely known and most used system for collecting and analyzing logs, metrics, and other data about the state of systems – servers, clusters, clouds.

It consists of three main components:

  • Elasticsearch: a database with quick search capabilities using the Elasticsearch Index
  • Logstash: a system for collecting data from various sources, transforming it, and transferring logs to Elasticsearch
  • Kibana: a web interface for displaying data from the Elasticsearch database

 

In addition, there is a set of so-called Beats for ELK (I’ll call it that out of habit), which are data collection utilities. Among them, for example, Filebeat – for collecting data from files (logs), or Metricbeat – for collecting data about the system – processor, RAM, etc.

The stack operation looks like this:

  • Server generates data, such as logs
  • The data is collected by the local Beat application, for logs it will be Filebeat (although this is an optional component and logs can be collected by Logstash itself), and sends it to Logstash or directly to Elastisearch
  • Logstash collects data from various sources (either from Beats or directly from Beats), transforms it as needed (adding or removing fields, tags, etc.), and sends it to Elasticsearch
  • Elasticsearch is designed to store data with the ability to quickly search
  • Kibana provides a web interface to work with Elasticsearch (and many other integrations)

What is Elasticsearch?

Elasticsearch is a distributed search and analysis system based on Apache Lucene. Shortly after its release in 2010, Elasticsearch became the most popular search engine and is commonly used for applications such as log analysis, full-text search, intelligent security, business intelligence, and process monitoring.

On January 21, 2021, Elastic NV announced a change in its software licensing strategy and that new versions of Elasticsearch and Kibana will not be released under the Apache License version 2.0 (ALv2) permissive license. Instead, new versions of the software are offered under the Elastic license, and the source code is available under the Elastic or SSPL license. These licenses are not open source and do not give users the same freedom. In an effort to provide open source professionals and our customers with a secure, high quality, fully open source search and analytics toolkit, we created the OpenSearch project, a community-developed offshoot of Elasticsearch and Kibana that is open source and licensed under the ALv2 license.

How does Elasticsearch work?

You can send data to Elasticsearch as JSON documents using APIs or ingestion tools such as Logstash and Amazon Kinesis Firehose. Elasticsearch automatically saves the original document and adds a link to it to the cluster index, including search capabilities. You can then search and retrieve the document using the Elasticsearch API. You can also use Kibana, a visualization tool with Elasticsearch, to visualize data and create interactive dashboards.

Versions of Elasticsearch licensed under the Apache 2.0 license (prior to 7.10.2 and Kibana 7.10.2) can be run on-premises, on Amazon EC2, or on Amazon OpenSearch. If you are deploying on-premises or on Amazon EC2, you are responsible for installing Elasticsearch and other required software, preparing the infrastructure, and managing the cluster. Amazon OpenSearch, on the other hand, is a fully managed service, so you don’t have to worry about the time-consuming process of managing clusters and tasks such as hardware preparation, software patching, disaster recovery, backup, and monitoring.

Advantages of Elasticsearch

Favorable price-to-performance ratio

Elasticsearch offers simple REST-based APIs and a lightweight HTTP interface, and uses schema-free JSON documents to get started and quickly build applications for a variety of use cases.

High performance

Elasticsearch’s distributed system allows you to process large amounts of data in parallel, instantly finding the best match for your query.

Free tools and modules

Elasticsearch is built into Kibana, a popular visualization and reporting tool. Integration with Beats and Logstash is also available, and source data can be easily converted and uploaded to the Elasticsearch cluster. You can use a number of open source Elasticsearch plugins, such as language analyzers and recommendation engines, to extend the functionality of your applications.

Real-time operations

Operations in Elasticsearch, such as reading or writing data, typically take less than a second. This allows you to use it in applications where you need to react in near real-time, such as application monitoring and anomaly detection.

Easy application development

Elasticsearch provides support for a variety of languages, including Java, Python, PHP, JavaScript, Node.js, Ruby, and many others.

 

What is Elastic Observability?

Transforming data to a standardized view – converging metrics, logs, and traces to provide a unified view.

 

What is Elastic Security

The Elastic Security platform enables analysts to prevent, detect, and respond to threats. Elastic Security supports Elastic Common Schema (ECS), a new specification that provides a consistent and customizable way to structure data in Elasticsearch, making it easier to analyze data from multiple sources. With ECS, you can use analytical content such as dashboards and machine learning tasks more broadly, search queries are narrower, and field names are easier to remember.

Elastic Security has a threat detection engine that automatically detects threats, reducing detection time and giving security teams more time to perform tasks that require expert input.

Elastic Security analyzes any data, automates key processes, and protects the OS. It is unified security on an open platform:

  • SIEM – Threat detection and rapid response in the cloud.
  • SOAR – Optimize SOC workflows with orchestration and automation.
  • Threat Intelligence – threat intelligence
  • EDR – Endpoint security
  • XDR – Power SecOps on your hosts, in the cloud, on the network, and beyond.
  • Cloud Security – Assess cloud health and protect your cloud workloads.