[jsr294-modularity-eg] EG response to 294 comments

[Bryan Atsatt]

Hi Alex,

I understand your points, but have a different take on them (surprise :^).

First, of course the compiler doesn't have to use modules, it would do 
so only when dependencies are expressed on them.

Second, a new 'module' access qualifier is clearly /not /public; it is a 
new mode more akin to default ('package private'). And, like default, I 
think it is better referred to in prose using more explicit terms, such 
as 'module private'.

Your position on the legacy compilation issue is interesting: a compile 
time warning of a /potential /runtime failure (the asymmetry alone sets 
off red flags for me). Under what circumstances would it /not /fail at 
runtime? Isn't the whole idea of a module private class that it cannot 
be accessed from outside the module, ever?

Like any other access violation, this should /fail /at compile time 
exactly as it will at runtime.

Changing an existing public class to module private is a potential 
breaking change. But this is no different than any other access 
reduction; it must be done with care.

Third, the legacy loader issue comes into play only when module private 
classes are packaged as .jar files ("simple module archives"). A legacy 
loader will not be able to deal with .jam files (yes, it is possible to 
configure a legacy loader to point at jars from an unpacked .jam file, 
but this is a clear encapsulation violation, similar to a non-container 
created loader in EE pointing at an unpacked .war, and should not be 
supported). So what do we do about module private classes and legacy 
loaders?

There is a fundamental question here, on which I think you and I may 
differ: should we support /two /overlapping but different module 
concepts (294 "modules" and 277 Modules)?

I don't think we should; the extra complexity and cognitive load is not 
worth it.

Instead, we should recognize that 277 Modules are (or will be) just as 
integral to Java as are ClassLoaders, and make them first class citizens 
as well (like it or not, the line between language and runtime is 
already blurred in this fashion).

This crystallizes for me when I think of java.lang.Class. It is very 
easy for me to grasp:

    public Module getModule(); // null if class is not a module member

But that doesn't work very well with the current proposal, so we have 
instead:

    public Superpackage getSuperpackage(); // null if class is not a 
superpackage member

But what about Module? Do we just leave it out? Or introduce both 
getModule() and getSuperpackage()? Or do we say that there is a subclass 
relationship, and you can downcast, sometimes? Shudder.

One module concept. One Class.getModule() method. Compile and runtime 
symmetry for both dependency resolution and access checks. No new binary 
for the JVM.

And /runtime /binding of class to Module, exactly as classes are 
currently bound to ClassLoaders for equality and package private access 
checks.

And yes, this means that a legacy loader calling defineClass() with a 
module private class requires some special handling. But this could be 
as simple as a generated Module bound to the ClassLoader instance. So 
/all /classes are bound to a Module at runtime, either provided or 
generated, and Class.getModule() can never return null. (Notice that 
this is quite easy to do in ClassLoader if we add a Module parameter to 
defineClass(): the existing methods create and cache the Module 
instance. This approach worked well in the 277 prototype I built.)

// Bryan 

Alex Buckley wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>   
>>  > All points from (A) thru (M) to (Z) recognize that a type's membership
>>  > of a superpackage is declared by a static artifact of the Java language.
>>  > This is essential because it makes reasoning about membership
>>  > straightforward. Any compile-time module system aimed at millions of
>>  > programmers cannot have module membership depend on artifacts outside
>>  > the language or on evaluation of arbitrary code.
>>
>> OSGi is living proof that this is not true.
>>     
>
> I went a bit too far. Having module membership depend on evaluation of 
> arbitrary code is clearly unreasonable if any semblance of readability 
> is to be maintained. Having module membership depend on static artifacts 
> is the only reasonable option; the question is where they live.
>
>   
>> OSGi assumes that the "module" is self-describing, and clearly does not 
>> depend on the language to do so; 277 could do the same. Membership could 
>> be as simple as deployment package containment, as it is in OSGi.
>>     
>
> It could be. But a compile-time module system aimed at millions of 
> programmers should, in the first instance, be thoroughly visible in the 
> language. A compilation unit whose types are members of a module should 
> document this fact. This is a moral position which has benefited the 
> Java language hugely over the years.
>
> In addition, let me present two scenarios which fall under the heading 
> of "migration compatibility". They recognize - as we had to with 
> generics - that not everyone will start using modules at the same time.
>
> 1) Legacy clients. If you compile a legacy client program which has no 
> knowledge of modules, and it accesses a public type which is to be 
> packaged in a module but not exported, the compilation MUST NOT give an 
> error. It is unacceptable to deny access to public types. But the legacy 
> client may now fail at run-time, which is unacceptable without some kind 
> of compile-time notification. Hence my proposal for a warning if a 
> non-exported public type is accessed at compile-time; it presages a 
> run-time failure in the same way an "unchecked" warning presages a 
> run-time failure due to heap pollution. (JLS 4.12.2.1)
>
> To give this warning, the language must be in charge of which module the 
> accessed public type is a member of.
>
> 2) Legacy classloaders. A legacy classloader running on JDK7 must 
> continue to be able to load classes even if those classes are members of 
>   module. To break such classloaders is unacceptable. This implies that 
> classes must be able to be members of a module without being packaged in 
> a module. It also implies that the VM is solely responsible for run-time 
> access control. (Indeed, with 'module' accessibility for members, *only* 
> the VM can possibly perform access control.)
>
> To support these classloaders, the classfile must be in charge of which 
> module it is a member of.
>
> Finally, I would point out that if you package classes in a module 
> archive, then purely at the conceptual level it seems reasonable for the 
> classfiles to claim membership of that module. Under what conditions 
> would the classfiles claim membership of another module?
>
>   
>> But we want to add compile-time access checks, and so, by definition, 
>> the language must support them in some fashion. But what do we really 
>> need here?
>>
>> (You may recall a document I sent around prior to the official formation 
>> of this JSR that described "module private" semantics; I'd re-send it 
>> here for context but no longer have it due to a drive crash plus an IT 
>> dept. backup disaster.)
>>
>> For both compilation and runtime we need two functions:
>>
>> 1. Membership: given a class and some artifact, determine its module.
>> 2. Access: given a class, determine if it is accessible to a class in 
>> another module.
>>
>> Exports can then be defined using only these functions: the set of all 
>> member classes which are accessible to other modules.
>>
>> The access function seems obvious, and is inline with your suggestion 
>> Andreas (and my original document): a 'module' qualifier which 
>> translates to an access flag in the class file.
>>
>> The membership function is clearly more interesting. At one extreme, we 
>> could add a module name declaration to each source file; the class 
>> itself becomes the artifact. At the other extreme we have a new 
>> compilation unit and artifact, which lists all class members.
>>
>> But both of these are brittle, and, just as bad, both ignore the version 
>> problem. Given the value of module versions at runtime, shouldn't they 
>> be just as important during compilation?
>>
>> I argued in my original document that there is a happy medium between 
>> these two extremes, one which supports versions during compilation 
>> EXACTLY as will the runtime:
>>
>> The compiler must USE the runtime.
>>
>> That is, module membership should be defined by an abstraction, and the 
>> compiler must use that abstraction. Given that pre-compiled dependencies 
>> exist in the form of modules, the module system itself provides that 
>> abstraction (ModuleDefinition) AND the access mechanism (Repository); 
>> the compiler must merely use them.
>>
>> Ok, so this makes sense once we know the dependencies, but... how do we 
>> define these for the compiler? Today we give it a list of jars called a 
>> classpath, so it seems to make sense that we could extend this to pass 
>> in a list of module name/version pairs. But this is not as flexible as 
>> the runtime, where version ranges and other constraints can be used to 
>> select dependencies, nor does it allow the compiler to implement the 
>> membership function for the source files.
>>
>> So, back to something like a "super-package.java" file, but this time 
>> lets support modules in a first class manner. Call it a "module.java" 
>> file, and have it contain:
>>
>> 1. A list of member *packages*, as in the current superpackage proposal.
>> 2. A list of import declarations, exactly as required by 277.
>>
>> The first enables the membership function; the second provides data for 
>> the compiler to call Repository methods to find dependencies.
>>
>> The module file can contain any additional annotations required by 277 
>> (or any other module system). The binary form can then be used by a tool 
>> to package up a .jam file (or an OSGi bundle with a bit more work).
>>     
>
> I agree with the majority of the above (though an explicit export list 
> is good for readability). I expect the 277 list will soon discuss how 
> javac could build modules and check their dependencies on other modules.
>
> Where I disagree is here: a Java compiler doesn't *have* to use modules. 
> It cannot demand that every type it compiles is a member of a module or 
> declares a dependency on a module or will be packaged in a module 
> archive. The language does not demand such things. Ironically, the 
> language does have to know about modules to handle legacy programs which 
> unknowingly interact with them (as in my first scenario for migration 
> compatibility) - but only to the degree of membership + access, not 
> import dependencies and versioning and so on.
>
>   
>> At runtime, the module system binds each class to a Module by passing 
>> the Module instance on the ClassLoader.defineClass() invocation.
>>
>> This model does not require that the .class files contain module name 
>> declarations, nor does it require a separate runtime binary solely for 
>> the JVM's consumption. Either or both could certainly be added, if this 
>> EG felt strongly that an extra level of membership enforcement is called 
>> for. I believe that the module packaging is sufficient to define 
>> membership, but others may disagree.
>>
>> And it doesn't required nesting, the complications of which I don't 
>> believe are worth the benefit.
>>     
>
> I believe I have stated in this mail why I disagree that module 
> packaging is sufficient to define membership. For now, I'm with you on 
> nesting.
>
> Alex
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>