Skip to content

test(dns): expand is_idn_hostname_uts46 coverage for the mapping step, Bidi and ContextO #2726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vtushar06 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

test(dns): expand is_idn_hostname_uts46 coverage for the mapping step, Bidi and ContextO #2726

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
267 changes: 267 additions & 0 deletions test/dns/idn_hostname_uts46_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#include <sourcemeta/core/dns.h>
#include <sourcemeta/core/test.h>

#include <string_view> // std::string_view

// A plain ASCII host name is unaffected by the mapping
TEST(valid_ascii_dotted) {
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46("www.example.com"));
Expand Down Expand Up @@ -116,3 +118,268 @@ TEST(invalid_trailing_dot) {
TEST(invalid_utf8) {
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46("\xc0\x80"));
}

// UTS #46 maps each U+FF41 to an ASCII letter, so the 63-codepoint label is 63
// octets after mapping even though it is 189 before
TEST(valid_length_measured_on_mapped_form) {
static constexpr char value[] =
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81";
static_assert(sizeof(value) - 1 == 189, "literal length");
const std::string_view input{value, 189};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// the same label one character longer is over the 63-octet limit
TEST(invalid_length_measured_on_mapped_form_at_64) {
static constexpr char value[] =
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81"
"\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81\xef\xbd\x81";
static_assert(sizeof(value) - 1 == 192, "literal length");
const std::string_view input{value, 192};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+200B is ignored, so it must not count toward the 63-octet label limit
TEST(valid_ignored_codepoint_not_counted_in_length) {
static constexpr char value[] = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
"aaaaaaaaaaaaaaaaaaa\xe2\x80\x8b";
static_assert(sizeof(value) - 1 == 66, "literal length");
const std::string_view input{value, 66};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// the ignored codepoint does not rescue a label that is already too long
TEST(invalid_ignored_codepoint_label_still_over) {
static constexpr char value[] = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
"aaaaaaaaaaaaaaaaaaaa\xe2\x80\x8b";
static_assert(sizeof(value) - 1 == 67, "literal length");
const std::string_view input{value, 67};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+FF58 U+FF4E map to "xn", so the xn-- test has to run after mapping
TEST(valid_ace_prefix_produced_by_mapping) {
static constexpr char value[] = "\xef\xbd\x98\xef\xbd\x8e--nxasmq6b";
static_assert(sizeof(value) - 1 == 16, "literal length");
const std::string_view input{value, 16};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// the mapped label is xn--b, which is not decodable Punycode
TEST(invalid_ace_prefix_produced_by_mapping_bad_payload) {
static constexpr char value[] = "\xef\xbd\x98\xef\xbd\x8e--b";
static_assert(sizeof(value) - 1 == 9, "literal length");
const std::string_view input{value, 9};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+FF0D maps to a hyphen, producing a leading hyphen
TEST(invalid_hyphen_produced_by_mapping_leading) {
static constexpr char value[] = "\xef\xbc\x8d"
"abc";
static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+FF0D maps to a hyphen, producing a trailing hyphen
TEST(invalid_hyphen_produced_by_mapping_trailing) {
static constexpr char value[] = "abc\xef\xbc\x8d";
static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// a pair of U+FF0D maps to -- in the third and fourth positions
TEST(invalid_double_hyphen_produced_by_mapping) {
static constexpr char value[] = "ab\xef\xbc\x8d\xef\xbc\x8d"
"cd";
static_assert(sizeof(value) - 1 == 10, "literal length");
const std::string_view input{value, 10};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// two U+FF0E map to full stops, producing an empty label
TEST(invalid_empty_label_produced_by_mapping) {
static constexpr char value[] = "a\xef\xbc\x8e\xef\xbc\x8e"
"b";
static_assert(sizeof(value) - 1 == 8, "literal length");
const std::string_view input{value, 8};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+03C2 is a deviation character; nontransitional processing keeps it
TEST(valid_deviation_final_sigma_kept) {
static constexpr char value[] = "\xcf\x82";
static_assert(sizeof(value) - 1 == 2, "literal length");
const std::string_view input{value, 2};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_TRUE(sourcemeta::core::is_idn_hostname(input));
}

// U+200D is a deviation character kept by nontransitional processing, and then
// fails its ContextJ rule
TEST(invalid_deviation_zwj_without_context) {
static constexpr char value[] = "a\xe2\x80\x8d"
"b";
static_assert(sizeof(value) - 1 == 5, "literal length");
const std::string_view input{value, 5};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+1D400 MATHEMATICAL BOLD CAPITAL A maps to a
TEST(valid_mathematical_bold_is_mapped) {
static constexpr char value[] = "\xf0\x9d\x90\x80"
"bc";
static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+FE00 VARIATION SELECTOR-1 is ignored by the mapping
TEST(valid_variation_selector_is_ignored) {
static constexpr char value[] = "a\xef\xb8\x80"
"b";
static_assert(sizeof(value) - 1 == 5, "literal length");
const std::string_view input{value, 5};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+24D0 CIRCLED LATIN SMALL LETTER A maps to a
TEST(valid_circled_latin_is_mapped) {
static constexpr char value[] = "\xe2\x93\x90"
"bc";
static_assert(sizeof(value) - 1 == 5, "literal length");
const std::string_view input{value, 5};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+FB01 LATIN SMALL LIGATURE FI maps to the two letters f and i
TEST(valid_ligature_is_mapped) {
static constexpr char value[] = "\xef\xac\x81x";
static_assert(sizeof(value) - 1 == 4, "literal length");
const std::string_view input{value, 4};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// U+005F is UTS #46 valid but carries the IDNA 2008 NV8 tag, so it is excluded
// from an IDN label
TEST(invalid_underscore_nv8) {
static constexpr char value[] = "a_b";
static_assert(sizeof(value) - 1 == 3, "literal length");
const std::string_view input{value, 3};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// the mapping step normalises to NFC, so the decomposed label is accepted
TEST(valid_nfc_composition_by_mapping) {
static constexpr char value[] = "cafe\xcc\x81";
static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// RFC 5893 condition 2: an RTL label may not contain a character with Bidi
// property L
TEST(invalid_bidi_ltr_char_inside_rtl_label) {
static constexpr char value[] = "\xd7\x90"
"a\xd7\x90";
static_assert(sizeof(value) - 1 == 5, "literal length");
const std::string_view input{value, 5};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// RFC 5893 condition 3: an RTL label must end with R, AL, EN, AN or NSM
TEST(invalid_bidi_rtl_label_ends_with_on) {
static constexpr char value[] = "\xd7\x90\xcb\x87";
static_assert(sizeof(value) - 1 == 4, "literal length");
const std::string_view input{value, 4};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// RFC 5893 condition 4: an RTL label may not contain both EN and AN
TEST(invalid_bidi_rtl_label_mixes_en_and_an) {
static constexpr char value[] = "\xd7\x90"
"0\xd9\xa0";
static_assert(sizeof(value) - 1 == 5, "literal length");
const std::string_view input{value, 5};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// RFC 5893 condition 1 allows only L, R or AL as a first character, and an
// Arabic-Indic digit is AN
TEST(invalid_bidi_pure_arabic_indic_digit_label) {
static constexpr char value[] = "\xd9\xa0\xd9\xa1";
static_assert(sizeof(value) - 1 == 4, "literal length");
const std::string_view input{value, 4};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// RFC 5892 appendix A.9 permits extended Arabic-Indic digits when the label
// carries none of the U+0660 block
TEST(valid_extended_arabic_indic_digits_only) {
static constexpr char value[] = "\xd8\xa8\xdb\xb0\xdb\xb1";
static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_TRUE(sourcemeta::core::is_idn_hostname(input));
}

// appendix A.9 forbids the two digit blocks in one label
TEST(invalid_extended_arabic_indic_mixed_with_arabic_indic) {
static constexpr char value[] = "\xd8\xa8\xdb\xb0\xd9\xa0";

@cubic-dev-ai cubic-dev-ai Bot Aug 14, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This test's label mixes U+06F0 (Bidi_Class EN) and U+0660 (Bidi_Class AN) after an RTL character, so it already fails RFC 5893 condition 4 (no mixing of EN and AN in an RTL label) independent of the Appendix A.9 digit-block rule. The comment claims this exercises A.9's prohibition on mixing digit blocks, but the failure here doesn't isolate that rule; use an LTR label or another approach that avoids triggering the bidi rule so the A.9 rule is the actual cause of rejection.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At test/dns/idn_hostname_uts46_test.cc, line 370:

<comment>This test's label mixes U+06F0 (Bidi_Class EN) and U+0660 (Bidi_Class AN) after an RTL character, so it already fails RFC 5893 condition 4 (no mixing of EN and AN in an RTL label) independent of the Appendix A.9 digit-block rule. The comment claims this exercises A.9's prohibition on mixing digit blocks, but the failure here doesn't isolate that rule; use an LTR label or another approach that avoids triggering the bidi rule so the A.9 rule is the actual cause of rejection.</comment>

<file context>
@@ -116,3 +118,268 @@ TEST(invalid_trailing_dot) {
+
+// appendix A.9 forbids the two digit blocks in one label
+TEST(invalid_extended_arabic_indic_mixed_with_arabic_indic) {
+  static constexpr char value[] = "\xd8\xa8\xdb\xb0\xd9\xa0";
+  static_assert(sizeof(value) - 1 == 6, "literal length");
+  const std::string_view input{value, 6};
</file context>

static_assert(sizeof(value) - 1 == 6, "literal length");
const std::string_view input{value, 6};
EXPECT_FALSE(sourcemeta::core::is_idn_hostname_uts46(input));

@augmentcode augmentcode Bot Aug 14, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At test/dns/idn_hostname_uts46_test.cc:373, this label contains U+06F0 (EN) and U+0660 (AN) after an RTL character, so it already fails RFC 5893 condition 4 even if Appendix A.9 is disabled. Consequently, this assertion cannot provide the claimed negative ContextO/A.9 coverage.

Severity: low

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

EXPECT_FALSE(sourcemeta::core::is_idn_hostname(input));
}

// appendices A.8 and A.9 scope the rule to "this label", so one block per label
// is permitted
TEST(valid_arabic_indic_digit_blocks_in_separate_labels) {
static constexpr char value[] = "\xd8\xa8\xd9\xa0.\xd8\xa8\xdb\xb0";
static_assert(sizeof(value) - 1 == 9, "literal length");
const std::string_view input{value, 9};
EXPECT_TRUE(sourcemeta::core::is_idn_hostname_uts46(input));
EXPECT_TRUE(sourcemeta::core::is_idn_hostname(input));
}
Loading