Repository navigation
Conversation
|
LGTM. @youkidearitai? |
|
@bukka @LamentXU123 Is this fine for PHP-8.6, given it's a bug fix with internals BC break - or should this target master? |
youkidearitai
left a comment
There was a problem hiding this comment.
I'm sorry for late.
My side looks good to me.
LamentXU123
left a comment
There was a problem hiding this comment.
From my perspective, I'd suggest to preserve the original macro sementics, and mergeup from 8.4 as a bugfix. And, merge the new macro changes (which cause the internal breaks) to master.
wdyt? @youkidearitai
|
@LamentXU123 I see. |
|
@youkidearitai Would you prefer me to create 4 seperate commit about this? |
There was a problem hiding this comment.
I think that we should target 8.4 with a minimal version of this PR (no behavior change for ZEND_LONG_UINT_OVFL(), less new macros).
Something I don't like about the existing macros is that they are limited to a small set of types, so in a few cases we check the wrong type (unsigned int instead of DWORD or uint32_t), relying on assumption that they are probably equal in practice.
As a follow up we should refactor to just one generic macro that accepts any integer type, in master:
ZEND_IN_RANGE_OF(value, DWORD);#define ZEND_TYPE_IS_SIGNED(T) ((T)-1 < 0)
#define ZEND_EXPR_IS_SIGNED(x) (0 * (x) - 1 < 0)
#define ZEND_TYPE_MAX(T) \
(ZEND_TYPE_IS_SIGNED(T) \
? (((((T)1 << (sizeof(T) * CHAR_BIT - 2)) - 1) * 2) + 1) \
: (T)-1)
#define ZEND_TYPE_MIN(T) \
(ZEND_TYPE_IS_SIGNED(T) ? -ZEND_TYPE_MAX(T) - 1 : (T)0)
#define ZEND_IN_RANGE_OF(x, T) \
(ZEND_TYPE_IS_SIGNED(T) \
? ((uintmax_t)ZEND_TYPE_MAX(T) <= INTMAX_MAX \
? (intmax_t)(x) >= (intmax_t)ZEND_TYPE_MIN(T) \
&& (intmax_t)(x) <= (intmax_t)ZEND_TYPE_MAX(T) \
: (x) >= 0) /* T is a uintmax_t-wide unsigned type */ \
: (uintmax_t)(x) <= (uintmax_t)ZEND_TYPE_MAX(T))| # define ZEND_LONG_EXCEEDS_INT(zlong) UNEXPECTED(ZEND_LONG_INT_OVFL(zlong) || ZEND_LONG_INT_UDFL(zlong)) | ||
| # define ZEND_LONG_UINT_OVFL(zlong) UNEXPECTED((zlong) < 0 || (zlong) > (zend_long)UINT_MAX) | ||
| # define ZEND_LONG_EXCEEDS_INT(zlong) UNEXPECTED((zlong) < (zend_long)INT_MIN || (zlong) > (zend_long)INT_MAX) | ||
| # define ZEND_LONG_UINT_OVFL(zlong) UNEXPECTED((zlong) > (zend_long)UINT_MAX) |
There was a problem hiding this comment.
We should keep the (zlong) < 0 check in case some external code relies on this behavior. All uses of ZEND_LONG_UINT_OVFL() in php-src relied on this.
|
Hi @arnaud-lb, nice idea actually - This can be used for
Additionally, these macros should start with The following should work /* Limits of an integer type, derived from the type itself.
All of them are integer constant expressions. */
#define ZEND_TYPE_IS_SIGNED(T) ((T)-1 < (T)0)
#define ZEND_TYPE_MAX(T) (ZEND_TYPE_IS_SIGNED(T) \
? (((((T)1 << (sizeof(T) * CHAR_BIT - 2)) - 1) * 2) + 1) \
: (T)-1)
#define ZEND_TYPE_MIN(T) (ZEND_TYPE_IS_SIGNED(T) ? -ZEND_TYPE_MAX(T) - 1 : (T)0)
/* zend_long vs. any integer type T.
Every ?: below is decided by T alone, so it is folded at compile time,
even without optimization. Where a check is not needed the whole macro is
a constant 0 or 1 and zlong is not evaluated, as with the former (0)
variants. Otherwise a single comparison remains, and only the selected
branch evaluates zlong, so zlong is never evaluated more than once.
A bound is only compared against when it is within the zend_long range,
so the casts of the bound to zend_long never change its value. */
/* zlong > max of T */
#define ZEND_LONG_OVFL(zlong, T) UNEXPECTED( \
(uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_LONG_MAX \
? (zend_long)(zlong) > (zend_long)ZEND_TYPE_MAX(T) \
: 0)
/* zlong < min of T */
#define ZEND_LONG_UDFL(zlong, T) UNEXPECTED( \
(intmax_t)ZEND_TYPE_MIN(T) > (intmax_t)ZEND_LONG_MIN \
? (zend_long)(zlong) < (zend_long)ZEND_TYPE_MIN(T) \
: 0)
/* min of T <= zlong <= max of T
With both bounds needed, a single unsigned comparison of the distance
from min against the width of the range checks both. */
#define ZEND_LONG_FITS(zlong, T) EXPECTED( \
(intmax_t)ZEND_TYPE_MIN(T) > (intmax_t)ZEND_LONG_MIN \
? ((uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_LONG_MAX \
? (zend_ulong)(zlong) - (zend_ulong)(zend_long)ZEND_TYPE_MIN(T) \
<= (zend_ulong)ZEND_TYPE_MAX(T) - (zend_ulong)(zend_long)ZEND_TYPE_MIN(T) \
: (zend_long)(zlong) >= (zend_long)ZEND_TYPE_MIN(T)) \
: ((uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_LONG_MAX \
? (zend_long)(zlong) <= (zend_long)ZEND_TYPE_MAX(T) \
: 1))
/* zlong < min of T || zlong > max of T
The exact negation of ZEND_LONG_FITS(). */
#define ZEND_LONG_EXCEEDS(zlong, T) UNEXPECTED( \
(intmax_t)ZEND_TYPE_MIN(T) > (intmax_t)ZEND_LONG_MIN \
? ((uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_LONG_MAX \
? (zend_ulong)(zlong) - (zend_ulong)(zend_long)ZEND_TYPE_MIN(T) \
> (zend_ulong)ZEND_TYPE_MAX(T) - (zend_ulong)(zend_long)ZEND_TYPE_MIN(T) \
: (zend_long)(zlong) < (zend_long)ZEND_TYPE_MIN(T)) \
: ((uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_LONG_MAX \
? (zend_long)(zlong) > (zend_long)ZEND_TYPE_MAX(T) \
: 0))
#define ZEND_ULONG_OVFL(zulong, T) UNEXPECTED( \
(uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_ULONG_MAX \
? (zend_ulong)(zulong) > (zend_ulong)ZEND_TYPE_MAX(T) \
: 0)
#define ZEND_ULONG_FITS(zulong, T) EXPECTED( \
(uintmax_t)ZEND_TYPE_MAX(T) < (uintmax_t)ZEND_ULONG_MAX \
? (zend_ulong)(zulong) <= (zend_ulong)ZEND_TYPE_MAX(T) \
: 1)
#define ZEND_ULONG_EXCEEDS(zulong, T) ZEND_ULONG_OVFL(zulong, T) |
|
Thank you for checking that code.
That was the problem: Replacing I didn't check this code thoroughly, however, so take this with caution.
I believe that the macro behaves as expected with any integer expression and type (as large as
Is this a functional issue or an optimization issue? The compiler would no-op the lower-bound check after a manual bound check, so it shouldn't add overhead.
We could probably overcome this: static inline bool in_range_s(intmax_t v, intmax_t min, uintmax_t max) {
if (max <= INTMAX_MAX) {
return v >= min && v <= (intmax_t)max;
}
return v >= 0; /* T is a uintmax_t-wide unsigned type */
}
static inline bool in_range_u(uintmax_t v, uintmax_t max) {
return v <= max;
}
#define ZEND_IN_RANGE_S(x, T) in_range_s((x), ZEND_TYPE_MIN(T), ZEND_TYPE_MAX(T))
#define ZEND_IN_RANGE_U(x, T) in_range_u((x), ZEND_TYPE_MAX(T))
#define ZEND_IN_RANGE_OF(x, T) _Generic((x) + 0, \
int: ZEND_IN_RANGE_S(x, T), \
long: ZEND_IN_RANGE_S(x, T), \
long long: ZEND_IN_RANGE_S(x, T), \
unsigned: ZEND_IN_RANGE_U(x, T), \
unsigned long: ZEND_IN_RANGE_U(x, T), \
unsigned long long: ZEND_IN_RANGE_U(x, T)) |
Seems to work for my quick and dirty tests.
Yea right - I somehow missed that and assumed it for
Mostly optimization for debug build. Some cases are checking one bound only (like https://github.com/php/php-src/blob/master/ext/bcmath/bcmath.c#L160) but these could be rewritten as well.
This should work in C11 but these macros are used in C++ as well (ext/intl) - where |
Yes I agree it would be useful
In that case I wouldn't bother, as the difference is really negligible |
46eb526 to
3251f2b
Compare
|
@arnaud-lb This PR now adds
Now, what belongs where? |
3251f2b to
a4a8259
Compare
arnaud-lb
left a comment
There was a problem hiding this comment.
Now, what belongs where?
For now I changed the target of this PR onto master - but as this is no longer touching the old macros, can this be targeting PHP-8.4 entirely or how should it be split?
This should target master only. For 8.4 we should commit a minimal fix (preferably before merging this PR).
Looks great otherwise.
a4a8259 to
81bdce7
Compare
|
@arnaud-lb |
|
Thank you @marc-mabe. I will give some time for code owners to review the split PRs and then will review/merge. |
| * The unary + applies the integer promotions, as (x) + 0 does for C. */ | ||
| extern "C++" { | ||
| template<typename T, typename X> | ||
| constexpr bool _zend_in_range_of(X x) { |
There was a problem hiding this comment.
that s the difference with the C counterpart, that might be not obvious if you do not pay attention. One of the possible case that could be an issue is where you use the expression inside static_assert for C/C++ regardless.
There was a problem hiding this comment.
might worth a noticeable comment maybe
There was a problem hiding this comment.
So your point is that static_assert(ZEND_IN_RANGE_OF(...)) works in C++ (as it's a constant expression) but not in C (error: “expression in static assertion is not constant”).
I can also replace the C inline functions with plain constant expressions, but that makes the signed variant more complex to avoid that x gets evaluated twice.
I think this should do the trick
# define _ZEND_IN_RANGE_SIGNED(x, T) ((((uintmax_t)(x) - (uintmax_t)ZEND_TYPE_MIN(T)) & ~(((uintmax_t)ZEND_TYPE_MAX(T) & INTMAX_MAX) - (uintmax_t)ZEND_TYPE_MIN(T))) == 0)
Actually, the unsigned variant would need to change as well to avoid warning comparison is always true due to limited range of data type
# define _ZEND_IN_RANGE_UNSIGNED(x, T) (((uintmax_t)(x) & ~(uintmax_t)ZEND_TYPE_MAX(T)) == 0)
@devnexen @arnaud-lb Should I change the C variant to constant expression or just add a comment to the C variant that this can not be used as constant expression?
There was a problem hiding this comment.
I think that it's fine if we can't use it in a static expression for now (I don't see a use-case where we will use ZEND_IN_RANGE_OF() with a constant value).
The only problem is that we might accidentally write code that compiles in C++ but not in C. Maybe just make the C++ version non-constexpr to avoid that?
There was a problem hiding this comment.
Updated (and rebased) the PR:
C and C++ variants now share the same inline helpers _zend_in_range_[signed|unsigned](), and the C++ template is no longer constexpr.
So ZEND_IN_RANGE_OF() now behaves the same in C and C++.
The existing range check macros each cover one pair of types, zend_long or size_t against int or unsigned int, so a caller narrowing into a DWORD, an uint32_t or an unsigned long has to pick the closest one and rely on the two types being of the same width. ZEND_IN_RANGE_OF(x, T) tells whether the value of the integer expression x is representable in the integer type T, for any integer x and T not wider than intmax_t, signed or unsigned. x is evaluated exactly once. In C it dispatches on the promoted type of x with _Generic; C++ has no _Generic, so there a function template deduces the type of x instead, which ext/intl needs. Both call the same inline functions doing the comparison in intmax_t or uintmax_t, so it behaves the same in C and C++, including not being a constant expression in either. Anything that is not an integer type, or is wider than intmax_t, fails to compile instead of silently truncating. ZEND_TYPE_MIN(T) and ZEND_TYPE_MAX(T), which it compares against, are public as well: the limits of any such integer type T as integer constant expressions, for types without a constant of their own such as time_t.
Move every caller of ZEND_LONG_INT_OVFL(), ZEND_LONG_INT_UDFL(), ZEND_LONG_EXCEEDS_INT(), ZEND_LONG_UINT_OVFL(), ZEND_SIZE_T_INT_OVFL() and ZEND_SIZE_T_UINT_OVFL() to ZEND_IN_RANGE_OF(). The old macros stay as they are for code outside php-src. Where the value is narrowed into another type than the one the old macro checked, check that type instead: a DWORD for the codepages in win32, whose messages now print MAXDWORD, the uint32_t the phar timestamp is stored in, and an int32_t where intl passes the value on to ICU and where cal_to_jd() limits the day, as their messages already say. onig_set_retry_limit_in_match_of_match_param() takes an unsigned long, so mbstring.regex_retry_limit is checked against that now. On LP64 a limit beyond the range of an unsigned int is passed on instead of being dropped in favour of oniguruma's default. Correct a message whose bounds did not match the check: the number of colors for imagetruecolortopalette() may be INT_MAX itself. Callers which deliberately leave the lower bound open keep doing so: bcmath accepts any negative precision, and in spec mode the DOM character data methods convert a negative offset or count as an unsigned long. This changes behaviour where ZEND_LONG_UINT_OVFL() expanded to (0), that is wherever a zend_long is not wider than an int: - Phar::buildFromIterator() now rejects a negative getMTime() override with a BadMethodCallException, as it does on 64bit. - sapi_windows_cp_set() and sapi_windows_cp_conv() now reject a negative codepage with a ValueError, as they do on 64bit. - a negative mbstring.regex_stack_limit or mbstring.regex_retry_limit is now ignored instead of being passed on as a huge unsigned int.
ZEND_LONG_INT_OVFL(), ZEND_LONG_INT_UDFL(), ZEND_LONG_EXCEEDS_INT(), ZEND_LONG_UINT_OVFL(), ZEND_SIZE_T_INT_OVFL() and ZEND_SIZE_T_UINT_OVFL() have no callers left in php-src. They stay for code outside of it, with their behaviour unchanged, but are deprecated in favour of ZEND_IN_RANGE_OF(). C has no way to deprecate a macro, so each of them calls the empty function _zend_deprecated_use_ZEND_IN_RANGE_OF_instead(), marked ZEND_ATTRIBUTE_DEPRECATED, instead. Every use then raises -Wdeprecated-declarations naming the replacement, and the compiler points at the macro. The flags ZEND_LONG_CAN_OVFL_INT, ZEND_LONG_CAN_OVFL_UINT and ZEND_SIZE_T_CAN_OVFL_UINT only serve these macros and are deprecated as well. Being only usable in #if, they can do so in a comment only. zend_portability.h includes zend_range_check.h last now, after ZEND_ATTRIBUTE_DEPRECATED is defined.
81bdce7 to
2b31859
Compare
Add ZEND_IN_RANGE_OF() and deprecate the zend_long/size_t vs. int range check macros
It adds a generic range check for any integer type, migrates every caller of the old range check macros to it and deprecates them. It deprecates but otherwise does not change the old macros.
The bug fixes found along the way are separate PRs against the lowest supported branch. They should be merged first; afterwards this PR gets rebased (tests and NEWS entries drop out of it)
ZEND_LONG_UINT_OVFL()accepting negative values on 32-bit platforms (negativembstring.regex_stack_limit/regex_retry_limit, negative codepage forsapi_windows_cp_set()/sapi_windows_cp_conv())Phar::buildFromIterator()with a negativegetMTime()on 32-bit (fixed by the PR above once merged up; the check only exists since 8.6)IntlDateFormatterparse functions truncating an$offsetbelowINT_MINimagetruecolortopalette()error message for$num_colorsZEND_IN_RANGE_OF(x, T)
Whether the value of the integer expression
xis representable in the integer typeT:xandTmay be any signed or unsigned integer types not wider thanintmax_t; anything else is a compile error (static assertion).xis evaluated exactly once._Genericon the promoted type ofx, comparing inintmax_toruintmax_t._Generic, so a function template deduces the type ofxthere.Also added:
ZEND_TYPE_MIN(T)andZEND_TYPE_MAX(T)as integer constant expressions, for types without a constant of their own.