Skip to content

Commit

Permalink
Merge pull request #3272 from opensearch-project/main
Browse files Browse the repository at this point in the history
[prod] OSCON session edits, speaker profile edits, data prepper roadmap update...
  • Loading branch information
nateynateynate authored Sep 5, 2024
2 parents 7f6b22f + 598c1d2 commit 8f1618c
Show file tree
Hide file tree
Showing 26 changed files with 439 additions and 231 deletions.
64 changes: 64 additions & 0 deletions _community_members/mvanjiappan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
#
# Sample data for community members.
#

# The short name is used to identify related pieces of content in the site. For example it is used in the "authors" array of blog posts, and it is used in the "presenters" array for OpenSearch Conference sessions to identify who is speaking.
# The format for existing values is not normalized. In some cases it is "first-initial-of-first-name" + "last-name", or matching a GitHub username, or something all together random. What is important is that it is unique within the system.
short_name: "mvanjiappan"

# The member's full name, or otherwise meaningful preferred name. It is used in the templates for presenting content authors as well as the name of conference speakers.
name: "Mohanraj Vanjiappan"

# The path to the community member's photo.
#photo: "/assets/media/..."

# Used as the level 1 page header text.
primary_title: 'Mohanraj Vanjiappan'

# Used as the document title displayed in the browser title bar.
title: 'OpenSearch Project Community Member: Mohanraj Vanjiappan'

# Community member job title and company where they work.
job_title_and_company: ''

# Array of conference IDs for which the community member is a keynote speaker, if any, or boolean false otherwise.
# This value is only relevant for member's with the "conference_speaker" user persona.
#keynote_speaker:
# - '2023-north-america'
# keynote_speaker: false

# The conference topic track for the conference sessions that the user is a speaker. These are shaped as an array of value pairs mapping conference ID and name.
# For example for the North American conference for the year 2023, and the "Community" track:
session_track:
- conference_id: "2024-north-america"
name: "Operating OpenSearch"

# URL for the community member profile.
permalink: '/community/members/vohanraj-vanjiappan.html'

# Array of community member user personas.
# At this time, only conference_speaker, and author are defined.
personas:
- conference_speaker

# Array of conference IDs that the member is a speaker.
conference_id:
- "2024-north-america"


# Optional GitHub username
#github: githubusername

# Optional LinkedIn username
#linkedin: linkedinusername

# Optional Twitter username
#twitter: twitterusername

# Optional Mastodon server url, and username / handle.
#mastodon:
# url: https://mastodon.social/@mastodonusername
# name: mastodonusername
---

3 changes: 2 additions & 1 deletion _community_members/rmanickam.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ primary_title: Ramesh Kumar Manickam
permalink: /community/members/rameshkumar-manickam.html
photo: "/assets/media/community/members/Ramesh.jpg"

job_title_and_company: "Sr Director at Transunion"
job_title_and_company: "Senior Engineering Manager, TransUnion Global Capability Center Bangalore"


conference_id:
Expand All @@ -22,3 +22,4 @@ personas:
linkedin: ramesh-kumar-manickam-60b90a14

---
Working as a Senior Engineering Manager At TransUnion Global Capability Center Bangalore. Solving problems in the domain of Risk & Fraud by leveraging cutting edge Technologies in digital landscape. Having 16+ years of IT Industry experience with hands on exposure to Java, Microservices, Google Clod Platform, Python, Data Science. Working in OpenSearch APIs, leveraging it to the fullest extend in the past couple of years.
5 changes: 2 additions & 3 deletions _community_members/rnair.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ short_name: "rnair"
name: "Rohit Nair"

# The path to the community member's photo.
photo: "/assets/media/community/members/no-image-available.svg"
photo: "/assets/media/community/members/rohitnair.jpeg"

# Used as the level 1 page header text.
primary_title: 'Rohit Nair'
Expand Down Expand Up @@ -63,5 +63,4 @@ conference_id:
# url: https://mastodon.social/@mastodonusername
# name: mastodonusername
---

Markdown or HTML content providing a user bio, or whatever is desired to describe their involvement in open source technology in general, and specifically how their work relates to the OpenSearch Project.
Rohit Nair is a Software Development Engineer at AWS currently working on OpenSearch Serverless. He has previously worked on areas like UltraWarm storage for Amazon OpenSearch Service and Amazon Kinesis Data Streams. He loves working at the intersection of distributed systems, scale and performance
4 changes: 2 additions & 2 deletions _data/top_nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,5 +67,5 @@ items:
url: https://playground.opensearch.org/
- label: Performance Benchmarks
url: /benchmarks/
- label: Release Metrics
url: /release-metrics/
- label: Release Dashboard
url: /release-dashboard/
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,14 @@ presenters:


---
The presentation will be slides only. The slides will entail different ways the customers can use the ABAC to achieve their solutions for their varied use-cases. This feature just a thought currently, going to raise a RFC this week and aligns with the RFC Theme for security. The ABAC can make the current FGAC much more dynamic based on multiple attributes -> temporal, spatial, role based, profile based, limit based, etc.
In the future it can be some language like OPA, etc. and then also be drag and drop based which can be configured graphically.
In this presentation, we will explore various industry-standard access control mechanisms and briefly cover the evolution of authorization. We'll focus on different access control methods, such as Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC), within the context of OpenSearch Security plugin's fine-grained access control features.

The Authorization engine also can be made more robust, extensive and decentralized with well known industry standards by introducing components like PAP(Policy Administration Point) - place where all the policies are kept and managed, PDP (Policy Decision Point) - place where the decision of a action is calculated or infered based on all the policies applicable for that user and resource with an ALLOW or DENY and finally PEP(Policy Enforcement Point) - this is the actual enforcement point which intercepts the incoming request and enforces the AuthZ based on above decision.
We'll delve into how the current features—including roles, document-level security, and field-level security with dynamic substitution—can be leveraged to address diverse use cases, such as segregating results based on entity attributes.

Additionally, we'll discuss future enhancements planned for the OpenSearch Security plugin. These proposed improvements aim to provide more robust context-based access control (ABAC) and expand support for a wider range of use cases.

By the end of this talk, attendees will have a comprehensive understanding of:
1. The evolution of access control mechanisms
2. Current OpenSearch Security plugin features and their applications
3. Upcoming developments in access control for OpenSearch

Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---

speaker_talk_title: Multi-Tenancy for all Workloads
speaker_talk_title: How AI/ML is changing information retrieval

primary_title: Multi-Tenancy for all Workloads
primary_title: How AI/ML is changing information retrieval

primary_presenter: jhandler

speaker_name_full: Jon Handler

title: 'OpenSearchCon 2024 North America Session: Multi-Tenancy for all Workloads'
title: 'OpenSearchCon 2024 North America Session: How AI/ML is changing information retrieval'

session_time: '2024-09-25 - 10:00am-10:40am'

session_room: 'Continental BR 1-3'

session_track: 'Operating OpenSearch'

permalink: '/events/opensearchcon/sessions/multitenancy-for-all-workloads.html'
permalink: '/events/opensearchcon/sessions/how-ai-ml-is-changing-information-retrieval.html'

#youtube_video_id: 'SOME_YOUTUBE_VIDEO_ID'

Expand All @@ -28,5 +28,4 @@ presenters:


---
All OpenSearch workloads are multi-tenant! Search workloads with multiple indices, languages, or customers, or logs workloads with multiple data streams, are multi-tenant. Handling that tenancy means aligning resources against data for optimal performance. Learn the strategies and solutions! I’ll cover the three main tenancy models - siloed, pooled, and hybrid. I’ll cover how to implement them, and tradeoffs with each model. At large scale, tenancy becomes a burning issue. I’ll talk through how to break up large, multi-tenant workloads, with the right architectures and tenant distribution. I’ll also cover operationalizing your tenancy strategy with routing, isolation, tenant management, cost and resource allocation.

When we work with information systems, we capture information in text and find information with text queries. Advances in AI have made it possible to move from word-to-word matching to something like meaning-to-meaning matching. Learn how search and OpenSearch unlock the meaning in your information. This session is foundational, covering why we search, and how we search to retrieve the best results. I will cover the core search algorithm, BM25 scoring, vectors (dense and sparse), LLMs, embedding generation and the neural and kNN plugins, exact, and approximate scoring.
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ permalink: '/events/opensearchcon/sessions/scaling-operational-intelligence-into
conference_id: '2024-north-america'

presenters:
- rmanickam
- rmanickam
- mvanjiappan



Expand Down
6 changes: 3 additions & 3 deletions _posts/2021-09-16-data-prepper-roadmap.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ redirect_from: "/blog/releases/2021/09/data-prepper-roadmap/"

*This blog has been updated for technical accuracy on 17 Nov 2022.*

The [roadmap](https://github.com/opensearch-project/data-prepper/projects/1) for Data Prepper is publicly available. Data Prepper is a component of the OpenSearch Project that accepts, filters, transforms, enriches, and routes data at scale. It is a key component of the trace analytics feature, which helps developers use distributed trace data to find and fix performance problems in distributed applications. The roadmap outlines a plan to enable Data Prepper to ingest logs and metrics from telemetry data collection agents, such as [Fluent Bit](https://fluentbit.io/) and the [Open Telemetry Collector](https://opentelemetry.io/docs/collector/). Data Prepper is the single data ingestion component for log and trace data pipelines that can scale to handle stateful processing of complex events such as trace data, aggregation transforms, and log-to-metric calculations.
The [roadmap](https://github.com/orgs/opensearch-project/projects/221) for Data Prepper is publicly available. Data Prepper is a component of the OpenSearch Project that accepts, filters, transforms, enriches, and routes data at scale. It is a key component of the trace analytics feature, which helps developers use distributed trace data to find and fix performance problems in distributed applications. The roadmap outlines a plan to enable Data Prepper to ingest logs and metrics from telemetry data collection agents, such as [Fluent Bit](https://fluentbit.io/) and the [Open Telemetry Collector](https://opentelemetry.io/docs/collector/). Data Prepper is the single data ingestion component for log and trace data pipelines that can scale to handle stateful processing of complex events such as trace data, aggregation transforms, and log-to-metric calculations.

### Data Prepper Overview

Expand All @@ -38,14 +38,14 @@ Data Prepper supports trace analytics data processing, as shown in Figure 2, and

### Roadmap

The [roadmap](https://github.com/opensearch-project/data-prepper/projects/1) for Data Prepper is published in the Data Prepper OpenSearch Project repository on GitHub. It gives you visibility on plans for new features, enhancements, and bug fixes, and we look forward to your feedback on it. In late 2021, the focus is on log collection from multiple sources through Data Prepper where they can be ingested, viewed, and analyzed in OpenSearch Dashboards. The [Data Prepper 1.2 release](https://opensearch.org/blog/technical-post/2021/12/Introducing-Data-Prepper-1.2.0-with-Log-Pipelines/) provides users the ability to send logs from Fluent Bit to OpenSearch or Amazon OpenSearch Service and use Grok to enhance the logs. These logs can then be correlated to traces coming from the OTEL Collectors to further enhance deep diving into your service problems using OpenSearch Dashboards. Below are the key features that we are targeting for the Data Prepper 1.2 release:
The [roadmap](https://github.com/orgs/opensearch-project/projects/221) for Data Prepper is published in the Data Prepper OpenSearch Project repository on GitHub. It gives you visibility on plans for new features, enhancements, and bug fixes, and we look forward to your feedback on it. In late 2021, the focus is on log collection from multiple sources through Data Prepper where they can be ingested, viewed, and analyzed in OpenSearch Dashboards. The [Data Prepper 1.2 release](https://opensearch.org/blog/technical-post/2021/12/Introducing-Data-Prepper-1.2.0-with-Log-Pipelines/) provides users the ability to send logs from Fluent Bit to OpenSearch or Amazon OpenSearch Service and use Grok to enhance the logs. These logs can then be correlated to traces coming from the OTEL Collectors to further enhance deep diving into your service problems using OpenSearch Dashboards. Below are the key features that we are targeting for the Data Prepper 1.2 release:

* HTTP Source Plugin (support for Fluent Bit)
* Grok processor
* Logstash template support for Grok
* Sample code and tutorials on creating processors

If you want to add something that is not in the [public roadmap](https://github.com/opensearch-project/data-prepper/projects/1) for OpenSearch Data Prepper, that’s a perfect opportunity to contribute! We are looking for contributors to help develop new processors and new source plugins and accelerate development of processors and source plugins on the roadmap. This public roadmap follows the same principles in the blog post outlining the [OpenSearch public roadmap](https://opensearch.org/blog/update/2021/05/opensearch-roadmap-announcement/). It is worth emphasizing a few points:
If you want to add something that is not in the [public roadmap](https://github.com/orgs/opensearch-project/projects/221) for OpenSearch Data Prepper, that’s a perfect opportunity to contribute! We are looking for contributors to help develop new processors and new source plugins and accelerate development of processors and source plugins on the roadmap. This public roadmap follows the same principles in the blog post outlining the [OpenSearch public roadmap](https://opensearch.org/blog/update/2021/05/opensearch-roadmap-announcement/). It is worth emphasizing a few points:

1. Date and milestones reflect intentions rather than firm commitments. Dates may change as anyone learns more or encounters unexpected issues. The roadmap will help make changes more transparent so that anyone can plan around them accordingly.
2. You can create a [feature request](https://github.com/opensearch-project/data-prepper/issues/new/choose) in the relevant GitHub repo for the feature and socialize the request. A maintainer or someone else in the community may pick up this feature and work on it. As progress is made, the maintainers will help get the feature onto the roadmap.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ With Data Prepper 1.2.0 out, the team is already working on
[Data Prepper 1.3.0](https://github.com/opensearch-project/data-prepper/milestone/2).
This release will focus on more ways to enrich log data. Current community needs drive the list of prioritized features.

What do you want to see in Data Prepper? You can impact the [project roadmap](https://github.com/opensearch-project/data-prepper/projects/1)
What do you want to see in Data Prepper? You can impact the [project roadmap](https://github.com/orgs/opensearch-project/projects/221)
in a few ways. First, comment on or up-vote issues in GitHub which you find valuable. Second, if you
don’t see it on the roadmap, please [create a Feature request](https://github.com/opensearch-project/data-prepper/issues/new/choose).
Finally, Data Prepper is open-source and [open to contributions](https://github.com/opensearch-project/data-prepper/blob/main/CONTRIBUTING.md).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ has other important features coming. We’d especially like to highlight the fol
* Data Prepper will begin to support metrics thanks to a [community contribution](https://github.com/opensearch-project/data-prepper/pull/1154).
* Many users have asked for a way to route different Events to different Sinks. Data Prepper’s [Conditional Routing](https://github.com/opensearch-project/data-prepper/issues/1007) will allow users to route based on our new Data Prepper Expression syntax.

You can see the [Data Prepper roadmap](https://github.com/opensearch-project/data-prepper/projects/1) to see other
You can see the [Data Prepper roadmap](https://github.com/orgs/opensearch-project/projects/221) to see other
upcoming changes. If there are any features on the roadmap that you are most interested in, please comment on the
issue to help the team prioritize issues. You can also request any changes by creating a
[GitHub issue](https://github.com/opensearch-project/data-prepper/issues/new/choose). This project is open source
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ add the concept of codecs to Data Prepper.

## What's next

The [Data Prepper roadmap](https://github.com/opensearch-project/data-prepper/projects/1) is the best place to see what
The [Data Prepper roadmap](https://github.com/orgs/opensearch-project/projects/221) is the best place to see what
is coming next. The maintainers are working toward Data Prepper 2.0, which will include
[conditional routing](https://github.com/opensearch-project/data-prepper/issues/1007) of events and
[core peer forwarding](https://github.com/opensearch-project/data-prepper/issues/700) for log aggregations.
2 changes: 1 addition & 1 deletion _posts/2022-10-10-Announcing-Data-Prepper-2.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ redirect_from: "/blog/technical-post/2022/10/Announcing-Data-Prepper-2.0.0/"
---

The Data Prepper maintainers are proud to announce the release of Data Prepper 2.0. This release makes
[Data Prepper](https://github.com/opensearch-project/data-prepper/projects/1)
[Data Prepper](https://github.com/orgs/opensearch-project/projects/221)
easier to use and helps you improve your observability stack based on feedback from our users. Data Prepper 2.0 retains
compatibility with all current versions of OpenSearch.

Expand Down
2 changes: 1 addition & 1 deletion _posts/2023-02-07-Announcing-Data-Prepper-2.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,6 @@ See the [release notes](https://github.com/opensearch-project/data-prepper/relea
You can [download](https://opensearch.org/downloads.html) Data Prepper or install a Docker container from the OpenSearch Download & Get Started page. The maintainers encourage
all users to update to Data Prepper 2.1.0 on order to gain the improved stability, additional administrative options, and new features.

Additionally, we have already started working on Data Prepper 2.2. See the [roadmap](https://github.com/opensearch-project/data-prepper/projects/1) to learn more.
Additionally, we have already started working on Data Prepper 2.2. See the [roadmap](https://github.com/orgs/opensearch-project/projects/221) to learn more.


2 changes: 1 addition & 1 deletion _posts/2023-04-19-Announcing-Data-Prepper-2.2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ collection, giving Amazon OpenSearch Serverless users the ability to use Data Pr

* To download Data Prepper, see the [OpenSearch downloads](https://opensearch.org/downloads.html) page.
* For instructions on how to get started with Data Prepper, see [Getting started with Data Prepper](https://opensearch.org/docs/2.6/data-prepper/getting-started/).
* To learn more about the work in progress for Data Prepper 2.3, see the [Data Prepper roadmap](https://github.com/opensearch-project/data-prepper/projects/1).
* To learn more about the work in progress for Data Prepper 2.3, see the [Data Prepper roadmap](https://github.com/orgs/opensearch-project/projects/221).


## Thanks to our contributors!
Expand Down
2 changes: 1 addition & 1 deletion _posts/2023-05-25-Announcing-Data-Prepper-2.3.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ processor:

* To download Data Prepper, see the [OpenSearch downloads](https://opensearch.org/downloads.html) page.
* For instructions on how to get started with Data Prepper, see [Getting started with Data Prepper](https://opensearch.org/docs/latest/data-prepper/getting-started/).
* To learn more about the work in progress for Data Prepper 2.4, see the [Data Prepper roadmap](https://github.com/opensearch-project/data-prepper/projects/1).
* To learn more about the work in progress for Data Prepper 2.4, see the [Data Prepper roadmap](https://github.com/orgs/opensearch-project/projects/221).


## Thanks to our contributors!
Expand Down
Loading

0 comments on commit 8f1618c

Please sign in to comment.