Description
The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
Published: 2026-08-20
Score: 8.7 High
EPSS: < 1% Very Low
KEV: No
Impact: n/a
Action: n/a
AI Analysis

Impact

The TweetTokenizer regular expression, compiled into WORD_RE, includes an unbounded naked‑domain branch that can be partitioned in exponentially many ways when presented with input that alternates label separators but lacks a top‑level domain. Because the engine enumerates all these partitions before failing, a few kilobytes of such crafted text can cause the tokenizer to consume seconds to minutes of CPU time. Since the tokenizer is intended for processing untrusted social‑media text, any service that uses TweetTokenizer.tokenize or casual_tokenize on user‑supplied content will become stalled per request without authentication. The vulnerability is a classic example of the Defect 1333: Improper Regular Expression Validation, leading to a denial‑of‑service.

Affected Systems

NLTK versions 3.6.5 through 3.9.2, including the beta 3.9‑beta1 release, are affected. The issue is present in the function casual_tokenize in nltk.tokenize.casual.py and is observed whenever the default TweetTokenizer is used. Version 3.10.1 of NLTK corrects the regular expression by bounding the domain‑label repetition, preventing the catastrophic backtracking scenario.

Risk and Exploitability

The CVSS score of 8.7 indicates high severity. EPSS is not available, but lack of an EPSS score does not diminish the risk; the vulnerability was identified publicly in August 2026 and is not listed in CISA’s KEV catalog, implying no known large‑scale exploits yet. Based on the description, the likely attack vector is sending specially crafted input that imitates domain patterns but without a top‑level domain to any service that processes user text with TweetTokenizer or casual_tokenize. No authentication is required, making the attack trivial to trigger from outside. Because the effect is a complete stall of the tokenization routine, the vulnerable system may experience service interruption for each request, leading to denial of service.

Generated by OpenCVE AI on August 21, 2026 at 00:36 UTC.

Remediation

No vendor fix or workaround currently provided.

OpenCVE Recommended Actions

  • Upgrade NLTK to version 3.10.1 or later, which bounds the regex and removes the backtracking issue.
  • If an upgrade is not possible, limit or reject input strings that are unusually long or contain repeated domain‑label patterns before passing them to the tokenizer.
  • Implement a timeout or resource cap on tokenizer calls and monitor CPU usage; consider switching to a safer tokenizer implementation if high load is observed.

Generated by OpenCVE AI on August 21, 2026 at 00:36 UTC.

Tracking

Sign in to view the affected projects.

Advisories
Source ID Title
Github GHSA Github GHSA GHSA-qx2g-xrx7-vfh8 NLTK TweetTokenizer vulnerable to denial of service through catastrophic regex backtracking
History

Mon, 31 Aug 2026 12:15:00 +0000

Type Values Removed Values Added
References
Metrics threat_severity

None

threat_severity

Important


Fri, 21 Aug 2026 11:30:00 +0000

Type Values Removed Values Added
Metrics ssvc

{'options': {'Automatable': 'yes', 'Exploitation': 'poc', 'Technical Impact': 'partial'}, 'version': '2.0.3'}


Thu, 20 Aug 2026 22:15:00 +0000

Type Values Removed Values Added
Description The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
Title NLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
First Time appeared Nltk
Nltk nltk
Weaknesses CWE-1333
CPEs cpe:2.3:a:nltk:nltk:3.6.5:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.6.6:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.6.7:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.7:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.8.1:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.8:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.9.1:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.9.2:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.9.3:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.9:*:*:*:*:*:*:*
cpe:2.3:a:nltk:nltk:3.9:beta1:*:*:*:*:*:*
Vendors & Products Nltk
Nltk nltk
References
Metrics cvssV3_1

{'score': 7.5, 'vector': 'CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H'}

cvssV4_0

{'score': 8.7, 'vector': 'CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N'}


cve-icon MITRE

Status: PUBLISHED

Assigner: VulnCheck

Published:

Updated: 2026-08-21T11:20:57.674Z

Reserved: 2026-08-10T15:12:16.754Z

Link: CVE-2026-72818

cve-icon Vulnrichment

Updated: 2026-08-21T11:06:02.698Z

cve-icon NVD

Status : Received

Published: 2026-08-20T22:18:05.087

Modified: 2026-08-21T11:17:05.920

Link: CVE-2026-72818

cve-icon Redhat

Severity : Important

Publid Date: 2026-08-20T21:57:34Z

Links: CVE-2026-72818 - Bugzilla

cve-icon OpenCVE Enrichment

Updated: 2026-08-21T01:15:06Z

Weaknesses
  • CWE-1333

    Inefficient Regular Expression Complexity