Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filters to Running Compactions table in monitor #4986

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ $(document).ready(function () {
]
});

const hostnameColumnName = 'hostname';
const queueNameColumnName = 'queueName';
const tableIdColumnName = 'tableId';
const durationColumnName = 'duration';

// Create a table for running compactors
runningTable = $('#runningTable').DataTable({
"ajax": {
Expand All @@ -93,7 +98,8 @@ $(document).ready(function () {
}
],
"columns": [{
"data": "server"
"data": "server",
"name": hostnameColumnName
},
{
"data": "kind"
Expand All @@ -102,10 +108,12 @@ $(document).ready(function () {
"data": "status"
},
{
"data": "queueName"
"data": "queueName",
"name": queueNameColumnName
},
{
"data": "tableId"
"data": "tableId",
"name": tableIdColumnName
},
{
"data": "numFiles"
Expand All @@ -131,7 +139,8 @@ $(document).ready(function () {
"data": "lastUpdate"
},
{
"data": "duration"
"data": "duration",
"name": durationColumnName
},
{ // more column settings
"class": "details-control",
Expand All @@ -142,6 +151,127 @@ $(document).ready(function () {
]
});

function handleFilterKeyup(input, feedbackElement, columnName) {
if (isValidRegex(input) || input === '') { // if valid, apply the filter
feedbackElement.hide();
$(this).removeClass('is-invalid');
const isRegex = true;
const smartEnabled = false;
runningTable
.column(`${columnName}:name`)
.search(input, isRegex, smartEnabled)
.draw();
} else { // if invalid, show the warning
feedbackElement.show();
$(this).addClass('is-invalid');
}
}

$('#hostname-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#hostname-feedback'), hostnameColumnName);
});

$('#queue-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#queue-feedback'), queueNameColumnName);
});

$('#tableid-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#tableid-feedback'), tableIdColumnName);
});

$('#duration-filter').on('keyup', function () {
runningTable.draw();
});

// Clear Filters button handler
$('#clear-filters').on('click', function () {
$(this).prop('disabled', true); // disable the clear button

// set the filter inputs to empty and trigger the keyup event to clear the filters
$('#hostname-filter').val('').trigger('keyup');
$('#queue-filter').val('').trigger('keyup');
$('#tableid-filter').val('').trigger('keyup');
$('#duration-filter').val('').trigger('keyup');

$(this).prop('disabled', false); // re-enable the clear
});

// Custom filter function for duration
$.fn.dataTable.ext.search.push(function (settings, data, dataIndex) {
if (settings.nTable.id !== 'runningTable') {
DomGarguilo marked this conversation as resolved.
Show resolved Hide resolved
return true;
}

const durationColIndex = runningTable.column(`${durationColumnName}:name`).index();
const durationStr = data[durationColIndex];
const durationSeconds = parseDuration(durationStr);

const input = $('#duration-filter').val().trim();
if (input === '') {
$('#duration-feedback').hide();
return true;
}

const match = validateDurationInput(input);
if (!match) {
$('#duration-feedback').show();
return false;
}

$('#duration-feedback').hide();
const operator = match[1];
const value = parseInt(match[2]);
const unit = match[3];
const filterSeconds = convertToSeconds(value, unit);

switch (operator) {
case '>':
return durationSeconds > filterSeconds;
case '>=':
return durationSeconds >= filterSeconds;
case '<':
return durationSeconds < filterSeconds;
case '<=':
return durationSeconds <= filterSeconds;
default:
console.error(`Unexpected operator "${operator}" encountered in duration filter.`);
return true;
}
});

// Helper function to convert duration strings to seconds
function convertToSeconds(value, unit) {
switch (unit.toLowerCase()) {
case 's':
return value;
case 'm':
return value * 60;
case 'h':
return value * 3600;
case 'd':
return value * 86400;
default:
console.error(`Unexpected unit "${unit}" encountered in duration filter. Defaulting to seconds.`);
return value;
}
}

// Helper function to validate duration input. Makes sure that the input is in the format of '<operator> <value> <unit>'
function validateDurationInput(input) {
return input.match(/^([<>]=?)\s*(\d+)([smhd])$/i);
}

/**
* @param {number} durationStr duration in milliseconds
* @returns duration in seconds
*/
function parseDuration(durationStr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running this again and figured out why I was not seeing any data. I had some config in my browser that was sending /rest/ec/running request to /dev/null.

Once I fixed that I could see data and noticed compaction ages were in years. Looked into this and I think its a unit mismatch. RunningCompactionInfo.java has the following line

   duration = last.getCompactionAgeNanos();

This function is expecting millis and maybe its getting nanos.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea it looks like thats the case. I thought it was a bug with how I was creating the sample data. The bug might be present in 2.1 as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I didn't notice this. For me, the sample data seemed normal (nothing over a couple hrs)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed via #5007. @keith-turner @kevinrr888

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When experimenting with these changes I was using accumulo-testing cingest to generate compactions. I started 10 compactor processes locally and ran ingest for a bit. The compactions were normally really fast and would not show up reliably. After a bit of data had built up, I forced full table compactions to create longer running compactions that I could look at in the monitor.

// Assuming durationStr is in milliseconds
const milliseconds = parseInt(durationStr, 10);
const seconds = milliseconds / 1000;
return seconds;
}

// Create a table for compaction coordinator
coordinatorTable = $('#coordinatorTable').DataTable({
"ajax": {
Expand Down Expand Up @@ -343,3 +473,13 @@ function refreshRunning() {
// user paging is not reset on reload
ajaxReloadTable(runningTable);
}

// Helper function to validate regex
function isValidRegex(input) {
try {
new RegExp(input);
return true;
} catch (e) {
return false;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,57 @@
<div class="row">
<div class="col-xs-12">
<table id="runningTable" class="table caption-top table-bordered table-striped table-condensed">
<caption><span class="table-caption">Running Compactions</span>&nbsp;&nbsp;
<a href="javascript:refreshRunning();"><span style="font-size: 1.5em; color: black;" class="bi bi-arrow-repeat"></span></a>
<caption>
<div class="d-flex justify-content-between align-items-center mb-3">
<div>
<span class="table-caption">Running Compactions</span>&nbsp;&nbsp;
<a href="javascript:refreshRunning();">
<span style="font-size: 1.5em; color: black;" class="bi bi-arrow-repeat"></span>
</a>
</div>
</div>
<div class="accordion" id="filterAccordion">
<div class="accordion-item">
<h2 class="accordion-header" id="filterHeading">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#filterCollapse" aria-expanded="false" aria-controls="filterCollapse">
Filters
</button>
</h2>
<div id="filterCollapse" class="accordion-collapse collapse" aria-labelledby="filterHeading" data-bs-parent="#filterAccordion">
<div class="accordion-body">
<!-- Hostname Filter -->
<div class="mb-3">
<label for="hostname-filter" class="form-label">Hostname Filter</label>
<input type="text" id="hostname-filter" class="form-control" placeholder="Enter hostname regex">
<small id="hostname-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<!-- Queue Filter -->
<div class="mb-3">
<label for="queue-filter" class="form-label">Queue Filter</label>
<input type="text" id="queue-filter" class="form-control" placeholder="Enter queue regex">
<small id="queue-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<!-- Table ID Filter -->
<div class="mb-3">
<label for="tableid-filter" class="form-label">Table ID Filter</label>
<input type="text" id="tableid-filter" class="form-control" placeholder="Enter table ID regex">
<small id="tableid-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<!-- Duration Filter -->
<div class="mb-3">
<label for="duration-filter" class="form-label">Duration Filter</label>
<input type="text" id="duration-filter" class="form-control" placeholder="Enter duration (e.g., &gt;10m, &lt;1h, &gt;=5s, &lt;=2d)">
<small id="duration-feedback" class="form-text text-danger" style="display:none;">Invalid duration format</small>
<small class="form-text text-muted">Valid formats: &gt;10m, &lt;1h, &gt;=5s, &lt;=2d</small>
</div>
<!-- Clear Filters Button -->
<div class="mb-3">
<button id="clear-filters" class="btn btn-secondary">Clear Filters</button>
</div>
</div>
</div>
</div>
</div>
</caption>
<thead>
<tr>
Expand Down