Clarification of the effective type restriction in C

Question

I am trying to understand the precise meaning of paragraphs 6 and 7 of Section 6.5 of the C18 Standard. Paragraph 6 defines the "effective type" of an object and 7 states in part "An object shall have its stored value accessed only by an lvalue expression that has one of the following types...a type compatible with the effective type of the object,...".

The following example is from an article on the C memory model. A function

short g(int *p, short *q) {
  short z = *q; *p = 10; return z;
}

is called in the following context:

union int_or_short { int x; short y; } u = { .y = 3 };
int *p = &u.x; // p points to the x variant of u
short *q = &u.y; // q points to the y variant of u
return g(p, q); // g is called with aliased pointers p and q

The author claims this exhibits undefined behavior. This is explained by considering an execution of the inlined version of the code:

union int_or_short { int x; short y; } u = { .y = 3 };
int *p = &u.x; // p points to the x variant of u
short *q = &u.y; // q points to the y variant of u
// g(p, q) is called, the body is inlined
short z = *q; // u has variant y and is accessed through y -> OK
*p = 10; // u has variant y and is accessed through x -> bad

The article states

The assignment *p = 10 violates the rules for effective types. The memory area where p points to contains a union whose variant is y of type short, but is accessed through a pointer to variant x of type int. This causes undefined behavior.

If I understand the argument correctly, the object pointed to by p (or q) has an effective type of short, and the last statement is accessing that object through an lvalue of type int, which is not compatible with short, hence a violation of paragraph 7 of the C Standard quoted above. But does the write *p=10 really count as an access to the stored value of the object?

Can someone explain, with reference to the language in the Standard, why, or why not, this code exhibits undefined behavior?

What about the following code:

union int_or_short { int x; short y; } u = { .y = 3 };
short z = u.y;
u.x = 10;

I always thought this code is perfectly legal (no undefined behavior), but if the first code is a violation, why isn't this one? Why would *p=10 count as an access to the stored value of the object and u.x=10 not? Both accesses are writes using an lvalue of type int.

Answer 1

Defect Report 028 was written between C89 and C99 to address the question about whether operations using pointers to union members should have the same semantics as operations performed on union members directly, and stated without justification that using pointers to perform actions which would be Implementation-Defined if performed on union members directly invoked Undefined Behavior. I think the Effective Type rule was intended to say that operations with pointers that would have standard-defined rather than implementation-defined behavior if those pointers happened to identify members of unions should have their behavior defined likewise, but it failed to say anything about the actual situation Defect Report 028 had been written to address.

What has always been needed to make type-based aliasing really work, but what the Standard has failed to articulate, is that it is intended to indicate when compilers must accommodate the possiblity that accesses that have no fresh visible relationship to something of a common type might alias. Given the code:

useFloatPtr(&someUnion.floatMember);
useUnsignedPtr(&someUnion.unsignedMember);
useFloatPtrAgain(&someUnion.floatMember);

a compiler that isn't being wilfully blind would have no trouble noticing that all three of the pointers passed to functions are freshly derived from a common type. If the functions are processed independently of the calling code, a compiler reason would have no reason to expect that the passed storage would be used elsewhere as something of a different type but also no reason to care. If the functions are in-lined, a compiler should be able to see the derivations of different pointer types that occur between the different function executions.

That's a very different situation from:

useFloatPtrAndUnsignedPtr(&someUnion.floatPtr, &someUnion.unsignedPtr);

where a compiler that generates code for that function may have reason to consolidate accesses via the float* across accesses via the unsigned*, and no reason to know it shouldn't do so.

Any implementation that was making a good faith effort to be compatible with existing programs and practices when the Standard was written would need to support the usage where each function would only access a union via pointer to one of its members, whether they did so as a consequence of not being able to in-line functions, or as a consequence of observing the derivation of union-member pointer types that occurred between the functions. It would have been awkward to write a standard in a way that would require implementations to recognize pointer derivations in circumstances when many implementations would have no reason to care about their absence, and the expenditure of ink would have helped neither in circumstances where compiler writers make a good faith effort to support existing practice, nor in circumstances where compiler writers do not make such a good faith effort.

Answer 2

Regarding

union int_or_short { int x; short y; } u = { .y = 3 };
int *p = &u.x;
short *q = &u.y;

, you write:

If I understand the article correctly, the object pointed to by p (or q) has an effective type of short,

If we interpret that as the article seems to mean, that p does not point to an int, then such a claim is not consistent with the C language spec. It seems to be trying to apply a rule from C17 6.5/6 that (loosely speaking) an object's effective type is determined by the type of the value most recent stored in it, but that rule does not apply here.

According to the spec,

The effective type of an object for an access to its stored value is the declared type of the object, if any.

(C17 6.5/6)

Pointer p points to u.x, which has declared type int. That is not changed by the fact that the storage for u.x overlaps the storage for u.y. Nor, for that matter, by the fact that the storage for both those objects overlaps the storage for u.

The rules about determining type by the value stored are later in that paragraph, but they apply only to objects having no declared type. The only objects contemplated by the language spec that do not have a declared type are dynamically allocated objects.

Perhaps the authors of the article got confused between allocated objects and objects accessed via pointers. Allocated objects can be accessed only via pointers, but not all accesses via pointers are to allocated objects. The rules for effective type are concerned with the nature of the object itself, not the means of access to it.

Moreover,

does the write *p=10 really count as an access to the stored value of the object?

[revised:]
Yes. "Access" is a defined term in the spec, meaning "to read or modify the value of an object" (C17 3.1/1). But that is not an issue. The effective type of *p is the declared type of the object to which it points, u.x, which is int. Furthermore, if, contrary to fact, we were talking about a dynamically allocated object, then the spec would have this to say about that assignment (still paragraph 6.5/6):

For all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.

So no, even if an allocated object were in play, there would be no issue with that assignment.

Of course, there is also no issue with performing the analogous assignment directly rather than via a pointer.

Answer 3

The paper’s author, Robbert Krebbers, does not assert the C code shown with g inlined has undefined behavior. That code is only provided to illustrate the inlining for the reader.

In a comment, you provided a link to the paper (which should have been provided in the question). Reading the paper, Krebbers talks about obtaining the code you show by inlining a function g into other code that calls it:

We will inline part of the function body of g to indicate the incorrect usage of aliased pointers during the execution of the example.

This transformation muddles the semantics—when the paper shows this new code with g inlined, it is not stated whether this new C code is intended to be reinterpreted as it appears, with the union definition visible when the inlined code from g is compiled or that this inlining is merely for display to the user of what happens when the code from g, with its original in situ interpretation, is moved into the place where g is called. In other words, in this latter case, the new code Krebbers shows after the function is inlined is merely a pseudo-code illustration of how inlining changes the code; it is not actual C code to be compiled.

The original g has parameters int *p, short *q, and no union declaration is visible. In this context, the compiler may assume *p and *q do not alias. At the point where g is called, the union declaration is visible, and it is apparent that u.x and u.y do alias. If we take the latter hypothesis above, that g is interpreted in situ, then everything is consistent:

• When compiling g, the compiler may assume *p and *q do not alias.
• It is an error to call g with pointers that cause aliasing. Even though, at the call site, the compiler can see that *p and *q alias, g was analyzed in an earlier context where this was not visible.
• The inlining of g brings the analyzed semantics of g (for example, the compiler’s internal representation of g) into the call site without reinterpreting the source code of g.

Given this, when Krebbers says that *p = 10; in the inlined code violates “the rules for effective types” (actually C 2018 6.5 7, often called the aliasing rule(s)), they are saying that the combination of the code of g with the pointers passed to it violates the aliasing rule. That is correct. Krebbers is only stating that the code in g has undefined behavior, not that the illustrative inlined code would have undefined behavior if it were compiled anew.