[jsr294-modularity-eg] EG response to 294 comments

[Bryan Atsatt]

Ugh. Please excuse the '/blah /' syntax; Thunderbird converted italics 
this way. I should know better :^)

Bryan Atsatt wrote:
> Hi Alex,
>
> I understand your points, but have a different take on them (surprise :^).
>
> First, of course the compiler doesn't have to use modules, it would do 
> so only when dependencies are expressed on them.
>
> Second, a new 'module' access qualifier is clearly /not /public; it is a 
> new mode more akin to default ('package private'). And, like default, I 
> think it is better referred to in prose using more explicit terms, such 
> as 'module private'.
>
> Your position on the legacy compilation issue is interesting: a compile 
> time warning of a /potential /runtime failure (the asymmetry alone sets 
> off red flags for me). Under what circumstances would it /not /fail at 
> runtime? Isn't the whole idea of a module private class that it cannot 
> be accessed from outside the module, ever?
>
> Like any other access violation, this should /fail /at compile time 
> exactly as it will at runtime.
>
> Changing an existing public class to module private is a potential 
> breaking change. But this is no different than any other access 
> reduction; it must be done with care.
>
> Third, the legacy loader issue comes into play only when module private 
> classes are packaged as .jar files ("simple module archives"). A legacy 
> loader will not be able to deal with .jam files (yes, it is possible to 
> configure a legacy loader to point at jars from an unpacked .jam file, 
> but this is a clear encapsulation violation, similar to a non-container 
> created loader in EE pointing at an unpacked .war, and should not be 
> supported). So what do we do about module private classes and legacy 
> loaders?
>
> There is a fundamental question here, on which I think you and I may 
> differ: should we support /two /overlapping but different module 
> concepts (294 "modules" and 277 Modules)?
>
> I don't think we should; the extra complexity and cognitive load is not 
> worth it.
>
> Instead, we should recognize that 277 Modules are (or will be) just as 
> integral to Java as are ClassLoaders, and make them first class citizens 
> as well (like it or not, the line between language and runtime is 
> already blurred in this fashion).
>
> This crystallizes for me when I think of java.lang.Class. It is very 
> easy for me to grasp:
>
>     public Module getModule(); // null if class is not a module member
>
> But that doesn't work very well with the current proposal, so we have 
> instead:
>
>     public Superpackage getSuperpackage(); // null if class is not a 
> superpackage member
>
> But what about Module? Do we just leave it out? Or introduce both 
> getModule() and getSuperpackage()? Or do we say that there is a subclass 
> relationship, and you can downcast, sometimes? Shudder.
>
> One module concept. One Class.getModule() method. Compile and runtime 
> symmetry for both dependency resolution and access checks. No new binary 
> for the JVM.
>
> And /runtime /binding of class to Module, exactly as classes are 
> currently bound to ClassLoaders for equality and package private access 
> checks.
>
> And yes, this means that a legacy loader calling defineClass() with a 
> module private class requires some special handling. But this could be 
> as simple as a generated Module bound to the ClassLoader instance. So 
> /all /classes are bound to a Module at runtime, either provided or 
> generated, and Class.getModule() can never return null. (Notice that 
> this is quite easy to do in ClassLoader if we add a Module parameter to 
> defineClass(): the existing methods create and cache the Module 
> instance. This approach worked well in the 277 prototype I built.)
>
> // Bryan 
>
> Alex Buckley wrote:
>   
>> Hi Bryan,
>>
>> Bryan Atsatt wrote:
>>   
>>     
>>>  > All points from (A) thru (M) to (Z) recognize that a type's membership
>>>  > of a superpackage is declared by a static artifact of the Java language.
>>>  > This is essential because it makes reasoning about membership
>>>  > straightforward. Any compile-time module system aimed at millions of
>>>  > programmers cannot have module membership depend on artifacts outside
>>>  > the language or on evaluation of arbitrary code.
>>>
>>> OSGi is living proof that this is not true.
>>>     
>>>       
>> I went a bit too far. Having module membership depend on evaluation of 
>> arbitrary code is clearly unreasonable if any semblance of readability 
>> is to be maintained. Having module membership depend on static artifacts 
>> is the only reasonable option; the question is where they live.
>>
>>   
>>     
>>> OSGi assumes that the "module" is self-describing, and clearly does not 
>>> depend on the language to do so; 277 could do the same. Membership could 
>>> be as simple as deployment package containment, as it is in OSGi.
>>>     
>>>       
>> It could be. But a compile-time module system aimed at millions of 
>> programmers should, in the first instance, be thoroughly visible in the 
>> language. A compilation unit whose types are members of a module should 
>> document this fact. This is a moral position which has benefited the 
>> Java language hugely over the years.
>>
>> In addition, let me present two scenarios which fall under the heading 
>> of "migration compatibility". They recognize - as we had to with 
>> generics - that not everyone will start using modules at the same time.
>>
>> 1) Legacy clients. If you compile a legacy client program which has no 
>> knowledge of modules, and it accesses a public type which is to be 
>> packaged in a module but not exported, the compilation MUST NOT give an 
>> error. It is unacceptable to deny access to public types. But the legacy 
>> client may now fail at run-time, which is unacceptable without some kind 
>> of compile-time notification. Hence my proposal for a warning if a 
>> non-exported public type is accessed at compile-time; it presages a 
>> run-time failure in the same way an "unchecked" warning presages a 
>> run-time failure due to heap pollution. (JLS 4.12.2.1)
>>
>> To give this warning, the language must be in charge of which module the 
>> accessed public type is a member of.
>>
>> 2) Legacy classloaders. A legacy classloader running on JDK7 must 
>> continue to be able to load classes even if those classes are members of 
>>   module. To break such classloaders is unacceptable. This implies that 
>> classes must be able to be members of a module without being packaged in 
>> a module. It also implies that the VM is solely responsible for run-time 
>> access control. (Indeed, with 'module' accessibility for members, *only* 
>> the VM can possibly perform access control.)
>>
>> To support these classloaders, the classfile must be in charge of which 
>> module it is a member of.
>>
>> Finally, I would point out that if you package classes in a module 
>> archive, then purely at the conceptual level it seems reasonable for the 
>> classfiles to claim membership of that module. Under what conditions 
>> would the classfiles claim membership of another module?
>>
>>   
>>     
>>> But we want to add compile-time access checks, and so, by definition, 
>>> the language must support them in some fashion. But what do we really 
>>> need here?
>>>
>>> (You may recall a document I sent around prior to the official formation 
>>> of this JSR that described "module private" semantics; I'd re-send it 
>>> here for context but no longer have it due to a drive crash plus an IT 
>>> dept. backup disaster.)
>>>
>>> For both compilation and runtime we need two functions:
>>>
>>> 1. Membership: given a class and some artifact, determine its module.
>>> 2. Access: given a class, determine if it is accessible to a class in 
>>> another module.
>>>
>>> Exports can then be defined using only these functions: the set of all 
>>> member classes which are accessible to other modules.
>>>
>>> The access function seems obvious, and is inline with your suggestion 
>>> Andreas (and my original document): a 'module' qualifier which 
>>> translates to an access flag in the class file.
>>>
>>> The membership function is clearly more interesting. At one extreme, we 
>>> could add a module name declaration to each source file; the class 
>>> itself becomes the artifact. At the other extreme we have a new 
>>> compilation unit and artifact, which lists all class members.
>>>
>>> But both of these are brittle, and, just as bad, both ignore the version 
>>> problem. Given the value of module versions at runtime, shouldn't they 
>>> be just as important during compilation?
>>>
>>> I argued in my original document that there is a happy medium between 
>>> these two extremes, one which supports versions during compilation 
>>> EXACTLY as will the runtime:
>>>
>>> The compiler must USE the runtime.
>>>
>>> That is, module membership should be defined by an abstraction, and the 
>>> compiler must use that abstraction. Given that pre-compiled dependencies 
>>> exist in the form of modules, the module system itself provides that 
>>> abstraction (ModuleDefinition) AND the access mechanism (Repository); 
>>> the compiler must merely use them.
>>>
>>> Ok, so this makes sense once we know the dependencies, but... how do we 
>>> define these for the compiler? Today we give it a list of jars called a 
>>> classpath, so it seems to make sense that we could extend this to pass 
>>> in a list of module name/version pairs. But this is not as flexible as 
>>> the runtime, where version ranges and other constraints can be used to 
>>> select dependencies, nor does it allow the compiler to implement the 
>>> membership function for the source files.
>>>
>>> So, back to something like a "super-package.java" file, but this time 
>>> lets support modules in a first class manner. Call it a "module.java" 
>>> file, and have it contain:
>>>
>>> 1. A list of member *packages*, as in the current superpackage proposal.
>>> 2. A list of import declarations, exactly as required by 277.
>>>
>>> The first enables the membership function; the second provides data for 
>>> the compiler to call Repository methods to find dependencies.
>>>
>>> The module file can contain any additional annotations required by 277 
>>> (or any other module system). The binary form can then be used by a tool 
>>> to package up a .jam file (or an OSGi bundle with a bit more work).
>>>     
>>>       
>> I agree with the majority of the above (though an explicit export list 
>> is good for readability). I expect the 277 list will soon discuss how 
>> javac could build modules and check their dependencies on other modules.
>>
>> Where I disagree is here: a Java compiler doesn't *have* to use modules. 
>> It cannot demand that every type it compiles is a member of a module or 
>> declares a dependency on a module or will be packaged in a module 
>> archive. The language does not demand such things. Ironically, the 
>> language does have to know about modules to handle legacy programs which 
>> unknowingly interact with them (as in my first scenario for migration 
>> compatibility) - but only to the degree of membership + access, not 
>> import dependencies and versioning and so on.
>>
>>   
>>     
>>> At runtime, the module system binds each class to a Module by passing 
>>> the Module instance on the ClassLoader.defineClass() invocation.
>>>
>>> This model does not require that the .class files contain module name 
>>> declarations, nor does it require a separate runtime binary solely for 
>>> the JVM's consumption. Either or both could certainly be added, if this 
>>> EG felt strongly that an extra level of membership enforcement is called 
>>> for. I believe that the module packaging is sufficient to define 
>>> membership, but others may disagree.
>>>
>>> And it doesn't required nesting, the complications of which I don't 
>>> believe are worth the benefit.
>>>     
>>>       
>> I believe I have stated in this mail why I disagree that module 
>> packaging is sufficient to define membership. For now, I'm with you on 
>> nesting.
>>
>> Alex
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>>   
>>     
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>