[FLINK-37925][table] Define VARIANT cast rules so a cast never alters the stored value

docs/content.zh/docs/sql/reference/data-types.md

@@ -1086,7 +1086,7 @@ The type can be declared using the above combinations where `p1` is the number o
 and `9` (both inclusive). If no `p1` is specified, it is equal to `2` by default. If no `p2` is
 specified, it is equal to `6` by default.
 
-### Constructured Data Types
+### Constructed Data Types
 
 #### `ARRAY`
 
@@ -1507,19 +1507,86 @@ close to the semantics of JSON. Compared to `ROW` and `STRUCTURED` type, `VARIAN
 flexibility to support highly nested and evolving schema.
 
 `VARIANT` allows for deeply nested data structures, such as arrays within arrays, maps within maps,
-or combinations of both.This capability makes `VARIANT` ideal for scenarios where data complexity
+or combinations of both. This capability makes `VARIANT` ideal for scenarios where data complexity
 and nesting are significant.
 
 `VARIANT` allows schema evolution, enabling the storage of data with changing or unknown schemas
 without requiring upfront schema definition. For example, if a new field is added to the data, it
 can be directly incorporated into the `VARIANT` data without modifying the table schema. This is
 particularly useful in dynamic environments where schemas may evolve over time.
 
-A primitive-valued `VARIANT` can be converted to a scalar type with `CAST` or `TRY_CAST`. Numeric
-targets are lenient: a variant holding any numeric value casts to any numeric type, so a JSON integer
-such as `PARSE_JSON('42')` casts to `INT` or `BIGINT`. Other targets require the stored value to be of
-the matching kind. When the value cannot be converted, `CAST` fails the job and `TRY_CAST` returns
-`NULL`. Use the `JSON_STRING` function to obtain the JSON string representation of a `VARIANT`.
+A `VARIANT` stores a single value of one of the following kinds: `NULL`, `BOOLEAN`, `TINYINT`,
+`SMALLINT`, `INT`, `BIGINT`, `FLOAT`, `DOUBLE`, `DECIMAL` (up to precision 38), `STRING`, `DATE`,
+`TIMESTAMP`, `TIMESTAMP_LTZ`, `BYTES`, or a nested array or object. `TIMESTAMP` and `TIMESTAMP_LTZ`
+are stored with microsecond precision.
+
+The `PARSE_JSON` function produces only the kinds that JSON syntax can express:
+
+| JSON input                                          | Stored `VARIANT` kind                           |
+|-----------------------------------------------------|-------------------------------------------------|
+| `null`                                              | `NULL`                                          |
+| `true` / `false`                                    | `BOOLEAN`                                       |
+| Integer within the 64-bit signed range              | smallest of `TINYINT`/`SMALLINT`/`INT`/`BIGINT` |
+| Integer beyond 64-bit, up to 38 significant digits  | `DECIMAL`                                       |
+| Decimal in plain notation, up to precision/scale 38 | `DECIMAL`                                       |
+| Number in scientific notation, e.g. `1.5e3`         | `DOUBLE`                                        |
+| Number exceeding 38 digits of precision or scale    | `DOUBLE`                                        |
+| String                                              | `STRING`                                        |
+| Array                                               | array (elements encoded by the same rules)      |
+| Object                                              | object (values encoded by the same rules)       |
+
+Because JSON has no literal for them, `PARSE_JSON` never produces `FLOAT`, `DATE`, `TIMESTAMP`,
+`TIMESTAMP_LTZ`, or `BYTES`.
+
+The JSON specification has no `NaN` or infinity literals, so `PARSE_JSON('NaN')`,
+`PARSE_JSON('Infinity')`, and `PARSE_JSON('-Infinity')` fail. `PARSE_JSON('1e400')` fails as well
+because a value outside the `DOUBLE` range cannot be stored as a finite number. In all of these
+cases `TRY_PARSE_JSON` returns `NULL`.
+A `VARIANT` has no dedicated kind for these values. To keep one, store it as a JSON string and cast
+it back out, for example `CAST(CAST(PARSE_JSON('"Infinity"') AS STRING) AS FLOAT)`.
+
+A `VARIANT` can be converted to a scalar type with `CAST` or `TRY_CAST`. A cast succeeds only when
+the target holds the stored value without altering it, so a value is never wrapped, rounded,
+truncated, or padded to make it fit. Otherwise `CAST` fails and `TRY_CAST` returns `NULL`.
+
+| Stored kind     | Succeeds for                                         |
+|-----------------|------------------------------------------------------|
+| numeric kinds   | any numeric target that holds the value              |
+| `BOOLEAN`       | `BOOLEAN`                                            |
+| `DATE`          | `DATE`                                               |
+| `TIMESTAMP`     | `TIMESTAMP(p)` that keeps the fractional seconds     |
+| `TIMESTAMP_LTZ` | `TIMESTAMP_LTZ(p)` that keeps the fractional seconds |
+| `BYTES`         | `BINARY(n)`, `VARBINARY(n)`                          |
+| any scalar      | `STRING`, `CHAR(n)`, `VARCHAR(n)`                    |
+| `NULL`          | SQL `NULL` for any nullable target                   |
+
+The conditions above mean:
+
+- An **integer target** needs the value in range and without a fractional part, so
+  `PARSE_JSON('7.0')` reaches `INT` as `7` while `PARSE_JSON('7.2')` does not, and
+  `CAST(PARSE_JSON('1000') AS TINYINT)` fails instead of wrapping.
+- A **`DECIMAL`** target has to fit the precision and the scale. Trailing zeros may be appended, so
+  `42` reaches `DECIMAL(5, 2)` as `42.00`, but a scale that would have to round is rejected.
+- **`FLOAT`** and **`DOUBLE`** are approximate by definition, so they take any numeric kind and drop
+  decimal digits, rejecting only a magnitude out of range such as `1e40` to a `FLOAT`.
+- **fractional seconds** have to survive the target precision. A variant keeps microseconds, so
+  `.123` reaches `TIMESTAMP(3)` while `.123456` does not.
+- **`CHAR(n)`** and **`BINARY(n)`** require the exact length, **`VARCHAR(n)`** and **`VARBINARY(n)`**
+  at most `n`. Nothing is padded or truncated.
+
+To reach a type the table does not list, wrap the cast in a regular cast. Only the inner cast is a
+`VARIANT` cast, so the outer one applies the usual rules and may round, truncate, or overflow:
+
+```sql
+CAST(CAST(PARSE_JSON('1000') AS SMALLINT) AS TINYINT)     -- returns -24 (after overflow)
+CAST(CAST(PARSE_JSON('3.9') AS DECIMAL(2, 1)) AS INT)     -- returns 3 (truncated)
+```
+
+A cast to a character string renders the value exactly as a regular SQL cast of the stored kind
+would, so a boolean becomes `TRUE`, a timestamp uses the SQL format, and a `TIMESTAMP_LTZ` is shifted
+into the session time zone. Use `JSON_STRING` for the JSON representation instead, where a string
+stays quoted as `"foo"` and an object or array is serialized. A variant that stores a JSON `null`
+casts to SQL `NULL`.
 
 **Declaration**
 

docs/content/docs/sql/reference/data-types.md

@@ -1094,7 +1094,7 @@ The type can be declared using the above combinations where `p1` is the number o
 and `9` (both inclusive). If no `p1` is specified, it is equal to `2` by default. If no `p2` is
 specified, it is equal to `6` by default.
 
-### Constructured Data Types
+### Constructed Data Types
 
 #### `ARRAY`
 
@@ -1515,19 +1515,86 @@ close to the semantics of JSON. Compared to `ROW` and `STRUCTURED` type, `VARIAN
 flexibility to support highly nested and evolving schema.
 
 `VARIANT` allows for deeply nested data structures, such as arrays within arrays, maps within maps, 
-or combinations of both.This capability makes `VARIANT` ideal for scenarios where data complexity 
+or combinations of both. This capability makes `VARIANT` ideal for scenarios where data complexity 
 and nesting are significant.
 
 `VARIANT` allows schema evolution, enabling the storage of data with changing or unknown schemas 
 without requiring upfront schema definition. For example, if a new field is added to the data, it 
 can be directly incorporated into the `VARIANT` data without modifying the table schema. This is 
 particularly useful in dynamic environments where schemas may evolve over time.
 
-A primitive-valued `VARIANT` can be converted to a scalar type with `CAST` or `TRY_CAST`. Numeric 
-targets are lenient: a variant holding any numeric value casts to any numeric type, so a JSON integer 
-such as `PARSE_JSON('42')` casts to `INT` or `BIGINT`. Other targets require the stored value to be of 
-the matching kind. When the value cannot be converted, `CAST` fails the job and `TRY_CAST` returns 
-`NULL`. Use the `JSON_STRING` function to obtain the JSON string representation of a `VARIANT`.
+A `VARIANT` stores a single value of one of the following kinds: `NULL`, `BOOLEAN`, `TINYINT`,
+`SMALLINT`, `INT`, `BIGINT`, `FLOAT`, `DOUBLE`, `DECIMAL` (up to precision 38), `STRING`, `DATE`,
+`TIMESTAMP`, `TIMESTAMP_LTZ`, `BYTES`, or a nested array or object. `TIMESTAMP` and `TIMESTAMP_LTZ`
+are stored with microsecond precision.
+
+The `PARSE_JSON` function produces only the kinds that JSON syntax can express:
+
+| JSON input                                          | Stored `VARIANT` kind                           |
+|-----------------------------------------------------|-------------------------------------------------|
+| `null`                                              | `NULL`                                          |
+| `true` / `false`                                    | `BOOLEAN`                                       |
+| Integer within the 64-bit signed range              | smallest of `TINYINT`/`SMALLINT`/`INT`/`BIGINT` |
+| Integer beyond 64-bit, up to 38 significant digits  | `DECIMAL`                                       |
+| Decimal in plain notation, up to precision/scale 38 | `DECIMAL`                                       |
+| Number in scientific notation, e.g. `1.5e3`         | `DOUBLE`                                        |
+| Number exceeding 38 digits of precision or scale    | `DOUBLE`                                        |
+| String                                              | `STRING`                                        |
+| Array                                               | array (elements encoded by the same rules)      |
+| Object                                              | object (values encoded by the same rules)       |
+
+Because JSON has no literal for them, `PARSE_JSON` never produces `FLOAT`, `DATE`, `TIMESTAMP`,
+`TIMESTAMP_LTZ`, or `BYTES`.
+
+The JSON specification has no `NaN` or infinity literals, so `PARSE_JSON('NaN')`,
+`PARSE_JSON('Infinity')`, and `PARSE_JSON('-Infinity')` fail. `PARSE_JSON('1e400')` fails as well
+because a value outside the `DOUBLE` range cannot be stored as a finite number. In all of these
+cases `TRY_PARSE_JSON` returns `NULL`.
+A `VARIANT` has no dedicated kind for these values. To keep one, store it as a JSON string and cast
+it back out, for example `CAST(CAST(PARSE_JSON('"Infinity"') AS STRING) AS FLOAT)`.
+
+A `VARIANT` can be converted to a scalar type with `CAST` or `TRY_CAST`. A cast succeeds only when
+the target holds the stored value without altering it, so a value is never wrapped, rounded,
+truncated, or padded to make it fit. Otherwise `CAST` fails and `TRY_CAST` returns `NULL`.
+
+| Stored kind     | Succeeds for                                         |
+|-----------------|------------------------------------------------------|
+| numeric kinds   | any numeric target that holds the value              |
+| `BOOLEAN`       | `BOOLEAN`                                            |
+| `DATE`          | `DATE`                                               |
+| `TIMESTAMP`     | `TIMESTAMP(p)` that keeps the fractional seconds     |
+| `TIMESTAMP_LTZ` | `TIMESTAMP_LTZ(p)` that keeps the fractional seconds |
+| `BYTES`         | `BINARY(n)`, `VARBINARY(n)`                          |
+| any scalar      | `STRING`, `CHAR(n)`, `VARCHAR(n)`                    |
+| `NULL`          | SQL `NULL` for any nullable target                   |
+
+The conditions above mean:
+
+- An **integer target** needs the value in range and without a fractional part, so
+  `PARSE_JSON('7.0')` reaches `INT` as `7` while `PARSE_JSON('7.2')` does not, and
+  `CAST(PARSE_JSON('1000') AS TINYINT)` fails instead of wrapping.
+- A **`DECIMAL`** target has to fit the precision and the scale. Trailing zeros may be appended, so
+  `42` reaches `DECIMAL(5, 2)` as `42.00`, but a scale that would have to round is rejected.
+- **`FLOAT`** and **`DOUBLE`** are approximate by definition, so they take any numeric kind and drop
+  decimal digits, rejecting only a magnitude out of range such as `1e40` to a `FLOAT`.
+- **fractional seconds** have to survive the target precision. A variant keeps microseconds, so
+  `.123` reaches `TIMESTAMP(3)` while `.123456` does not.
+- **`CHAR(n)`** and **`BINARY(n)`** require the exact length, **`VARCHAR(n)`** and **`VARBINARY(n)`**
+  at most `n`. Nothing is padded or truncated.
+
+To reach a type the table does not list, wrap the cast in a regular cast. Only the inner cast is a
+`VARIANT` cast, so the outer one applies the usual rules and may round, truncate, or overflow:
+
+```sql
+CAST(CAST(PARSE_JSON('1000') AS SMALLINT) AS TINYINT)     -- returns -24 (after overflow)
+CAST(CAST(PARSE_JSON('3.9') AS DECIMAL(2, 1)) AS INT)     -- returns 3 (truncated)
+```
+
+A cast to a character string renders the value exactly as a regular SQL cast of the stored kind
+would, so a boolean becomes `TRUE`, a timestamp uses the SQL format, and a `TIMESTAMP_LTZ` is shifted
+into the session time zone. Use `JSON_STRING` for the JSON representation instead, where a string
+stays quoted as `"foo"` and an object or array is serialized. A variant that stores a JSON `null`
+casts to SQL `NULL`.
 
 **Declaration**
 

docs/data/sql_functions.yml

@@ -1234,6 +1234,8 @@ variant:
       parser will keep the last occurrence of all fields with the same key, otherwise when 
       `allowDuplicateKeys` is false it will throw an error. The default value of 
       `allowDuplicateKeys` is false.
+
+      See the `VARIANT` data type documentation for how JSON values are mapped to variant kinds.
   - sql: TRY_PARSE_JSON(json_string[, allow_duplicate_keys])
     description: |
       Try to parse a JSON string into a Variant if possible. If the JSON string is invalid, return 
@@ -1243,6 +1245,8 @@ variant:
       parser will keep the last occurrence of all fields with the same key, otherwise when 
       `allowDuplicateKeys` is false it will throw an error. The default value of 
       `allowDuplicateKeys` is false.
+
+      See the `VARIANT` data type documentation for how JSON values are mapped to variant kinds.
   - sql: JSON_STRING(variant)
     description: |
       Generate a json string from a Variant object.

docs/data/sql_functions_zh.yml

@@ -1320,6 +1320,8 @@ variant:
       同键的字段，否则当 allowDuplicateKeys 为 false 时，它会抛出一个错误。默认情况下，
       allowDuplicateKeys 的值为 false。
 
+      See the `VARIANT` data type documentation for how JSON values are mapped to variant kinds.
+
   - sql: TRY_PARSE_JSON(json_string[, allow_duplicate_keys])
     description: |
       尽可能将 JSON 字符串解析为 Variant。如果 JSON 字符串无效，则返回 NULL。如果希望抛出错误而不是返回 NULL，
@@ -1329,6 +1331,8 @@ variant:
       同键的字段，否则当 allowDuplicateKeys 为 false 时，它会抛出一个错误。默认情况下，
       allowDuplicateKeys 的值为 false。
 
+      See the `VARIANT` data type documentation for how JSON values are mapped to variant kinds.
+
   - sql: JSON_STRING(variant)
     description: |
       Generate a json string from a Variant object.

flink-core/src/main/java/org/apache/flink/types/variant/Variant.java

@@ -94,7 +94,8 @@ public interface Variant {
     float getFloat() throws VariantTypeException;
 
     /**
-     * Get the scalar value of variant as BigDecimal, if the variant type is {@link Type#DECIMAL}.
+     * Get the scalar value of variant as {@link BigDecimal}, if the variant type is {@link
+     * Type#DECIMAL}.
      *
      * @throws VariantTypeException If this variant is not a scalar value or is not {@link
      *     Type#DECIMAL}.
@@ -118,27 +119,29 @@ public interface Variant {
     String getString() throws VariantTypeException;
 
     /**
-     * Get the scalar value of variant as LocalDate, if the variant type is {@link Type#DATE}.
+     * Get the scalar value of variant as {@link LocalDate}, if the variant type is {@link
+     * Type#DATE}.
      *
      * @throws VariantTypeException If this variant is not a scalar value or is not {@link
      *     Type#DATE}.
      */
     LocalDate getDate() throws VariantTypeException;
 
     /**
-     * Get the scalar value of variant as LocalDateTime, if the variant type is {@link
-     * Type#TIMESTAMP}.
+     * Get the scalar value of variant as {@link LocalDateTime}, if the variant type is {@link
+     * Type#TIMESTAMP}. The returned value has microsecond precision.
      *
      * @throws VariantTypeException If this variant is not a scalar value or is not {@link
      *     Type#TIMESTAMP}.
      */
     LocalDateTime getDateTime() throws VariantTypeException;
 
     /**
-     * Get the scalar value of variant as Instant, if the variant type is {@link Type#TIMESTAMP}.
+     * Get the scalar value of variant as {@link Instant}, if the variant type is {@link
+     * Type#TIMESTAMP_LTZ}. The returned value has microsecond precision.
      *
      * @throws VariantTypeException If this variant is not a scalar value or is not {@link
-     *     Type#TIMESTAMP}.
+     *     Type#TIMESTAMP_LTZ}.
      */
     Instant getInstant() throws VariantTypeException;
 

flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/inference/strategies/CastInputTypeStrategy.java

@@ -35,7 +35,6 @@
 import java.util.List;
 import java.util.Optional;
 
-import static org.apache.flink.table.types.logical.utils.LogicalTypeCasts.getUnsupportedCastHint;
 import static org.apache.flink.table.types.logical.utils.LogicalTypeCasts.supportsExplicitCast;
 
 /**
@@ -70,15 +69,6 @@ public Optional<List<DataType>> inferInputTypes(
             return Optional.of(argumentDataTypes);
         }
         if (!supportsExplicitCast(fromType, toType)) {
-            final Optional<String> hint = getUnsupportedCastHint(fromType, toType);
-            if (hint.isPresent()) {
-                return callContext.fail(
-                        throwOnFailure,
-                        "Unsupported cast from '%s' to '%s'. %s",
-                        fromType,
-                        toType,
-                        hint.get());
-            }
             return callContext.fail(
                     throwOnFailure, "Unsupported cast from '%s' to '%s'.", fromType, toType);
         }