Release v0.29.1

IMPROVEMENTS

• feat: read WITH_EU_SCREENING_LIST as option to skip EU CSL list
• feat: usaddress: add Similarity function
• remove MySQL and sqlite dependencies, resolves #468
• remove bloat not needed any more when we build with CGO_ENABLED=0
• remove database from dependencies and build system

BUILD

• build(deps): bump body-parser and express in /webui
• build(deps): bump braces from 3.0.2 to 3.0.3 in /webui
• build(deps): bump ejs from 3.1.9 to 3.1.10 in /webui
• build(deps): bump express from 4.18.2 to 4.19.2 in /webui
• build(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /webui
• build(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0
• build(deps): bump micromatch from 4.0.5 to 4.0.8 in /webui
• build(deps): bump rexml from 3.2.6 to 3.3.6 in /docs
• build(deps): bump rollup from 2.79.1 to 2.79.2 in /webui
• build(deps): bump webpack from 5.89.0 to 5.95.0 in /webui
• build(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /webui
• build(deps): bump webrick from 1.8.1 to 1.8.2 in /docs
• build(deps-dev): bump nokogiri from 1.16.2 to 1.16.5 in /docs
• build: convert docker-compose to docker compose
• build: switch from docker-compose
• chore(deps): update dependency github-pages to v232
• fix(deps): update dependency express to v4.19.2 [security]
• fix(deps): update react monorepo to v18.3.1
• fix(deps): updates body-parser from 1.20.2 to 1.20.3
• fix(deps): updates express from 4.19.2 to 4.21.0

Release v0.28.2

IMPROVEMENTS

• fix: download all files if the initial dir does not exist

Release v0.28.1

IMPROVEMENTS

• fix: close and cleanup downloaded files after successful parsing

BUILD

• build(deps-dev): bump nokogiri from 1.13.10 to 1.16.2 in /docs
• chore(deps): update dependency github-pages to v231

Release v0.28.0

ADDITIONS

• feat: add /crypto endpoint and extract digital currency addresses from OFAC
• feat: add SEARCH_MAX_WORKERS

IMPROVEMENTS

• build: fix npm build for Openshift
• build: force latest stable Go when building docker images
• fix: accumulate SDN comments for merging overflow after all files are read
• fix: allocate known-capacity arrays
• fix: copy extended remarks from sdn_comments.csv
• largest: improve performance, simplify insert step

BUILD

• build(deps): bump golang.org/x/crypto from 0.15.0 to 0.17.0

Release v0.27.0

This release of Watchman includes additional improvements to the search match scores to reduce false positives and increase true positive matches. A few of the specific improvements are:

1. Compare tokens in the search to the index tokens
   • i.e. "find matches for every search token" rather than "find match for every indexed token"
   • Improves scores of searches that don't include "middle" names
   • Prevents sanctioned names that are 1 word (HADI, EMMA, KAMILA) matching long searches
   • Has a side-effect that short search terms will have more false positives. I think this is a good trade off as the sanction lists will always contain the full name, but the search might not
2. Once a token has matched something, it can't match a different token
   • This prevents names with repeated words having artificially high scores
   • e.g. prevents any search containing "Vladimir" matching "VLADIMIROV, Vladimir Vladimirovich"
3. Weights each word-score by the length of the word, relative to the search and indexed name
   • This corrects for error that is introduced by splitting names into tokens and doing piecewise Jaro-Winkler scoring
   • Combing word-scores using a simple average gives short words (like Li, Al) equal weight to much longer words
   • The length-weighted scores are comparable to what you get by doing whole-name to whole-name Jaro-Winkler comparison
4. Punishes word-scores when the matching tokens have significantly different length
5. Punishes word-scores when the matching tokens start with different letters

Release v0.26.1

This release of Watchman has removed Company/Customer models and Watches. They've been deprecated for a while and do not perform as users expect. Stay tuned for a future Moov OSS project integrating with Watchman for sanctions screening.

IMPROVEMENTS

• feat: return matchedName in non-OFAC results
• search: apply more edge case logic to decrease bad scoring
• search: return matchedName for OFAC SDNs, Alts, and DPL records
• test: remove duplicate (and skipped) UK/EU CSL tests

Release v0.25.0

This release of Watchman lowers most match percentages by comparing names better.

IMPROVEMENTS

• fix: close xml encoder
• fix: panic cleanup from newer linter rules
• cmd/server: only check adjacent terms for local jaro max score
• cmd/server: read ADJACENT_SIMILARITY_POSITIONS env var
• cmd/server: weight term score by length similarity

BUILD

• build: bump numerous javascript dependencies
• build: update to Go 1.21
• build: update to node 20
• build: update Debian, Fedora, node, and Go base images

Release v0.24.2

IMPROVEMENTS

• fix: keep numbers during stopwords step
• fix: stop setting level=error for info logs

BUILD

• bump golang.org/x/crypto to v0.6.0
• bump golang.org/x/net from 0.6.0 to 0.7.0
• build(deps): bump activesupport from 6.1.7.2 to 7.0.4.3 in /docs
• build(deps): bump webpack from 5.75.0 to 5.76.1 in /webui

Release v0.24.1

IMPROVEMENTS

• fix: filter SDNs in async search rather than truncate

BUILD

• build: update Go dependencies
• docs: bundle update
• webui: npm audit fix

Release v0.24.0

ADDITIONS

• search: Add the EU Consolidated Screening List
• search: Add the UK Consolidated Screening List

IMPROVEMENTS

• feat: log status after download
• fix: guard around race condition in pkg/download
• fix: cap match percentage

BUILD

• build: upgrade golang to 1.20
• build: try using hashicorp-forge/go-test-split-action to speedup tests
• build(deps): bump activesupport from 6.0.3.4 to 6.0.6.1 in /docs
• build(deps): bump json5 from 1.0.1 to 1.0.2 in /webui
• build(deps): bump loader-utils from 2.0.0 to 2.0.4 in /webui
• build(deps): bump nokogiri from 1.13.6 to 1.13.9 in /docs