[jsr294-modularity-eg] EG response to 294 comments

[Alex Buckley]

Hi Bryan,

Bryan Atsatt wrote:
>  > All points from (A) thru (M) to (Z) recognize that a type's membership
>  > of a superpackage is declared by a static artifact of the Java language.
>  > This is essential because it makes reasoning about membership
>  > straightforward. Any compile-time module system aimed at millions of
>  > programmers cannot have module membership depend on artifacts outside
>  > the language or on evaluation of arbitrary code.
> 
> OSGi is living proof that this is not true.

I went a bit too far. Having module membership depend on evaluation of 
arbitrary code is clearly unreasonable if any semblance of readability 
is to be maintained. Having module membership depend on static artifacts 
is the only reasonable option; the question is where they live.

> OSGi assumes that the "module" is self-describing, and clearly does not 
> depend on the language to do so; 277 could do the same. Membership could 
> be as simple as deployment package containment, as it is in OSGi.

It could be. But a compile-time module system aimed at millions of 
programmers should, in the first instance, be thoroughly visible in the 
language. A compilation unit whose types are members of a module should 
document this fact. This is a moral position which has benefited the 
Java language hugely over the years.

In addition, let me present two scenarios which fall under the heading 
of "migration compatibility". They recognize - as we had to with 
generics - that not everyone will start using modules at the same time.

1) Legacy clients. If you compile a legacy client program which has no 
knowledge of modules, and it accesses a public type which is to be 
packaged in a module but not exported, the compilation MUST NOT give an 
error. It is unacceptable to deny access to public types. But the legacy 
client may now fail at run-time, which is unacceptable without some kind 
of compile-time notification. Hence my proposal for a warning if a 
non-exported public type is accessed at compile-time; it presages a 
run-time failure in the same way an "unchecked" warning presages a 
run-time failure due to heap pollution. (JLS 4.12.2.1)

To give this warning, the language must be in charge of which module the 
accessed public type is a member of.

2) Legacy classloaders. A legacy classloader running on JDK7 must 
continue to be able to load classes even if those classes are members of 
  module. To break such classloaders is unacceptable. This implies that 
classes must be able to be members of a module without being packaged in 
a module. It also implies that the VM is solely responsible for run-time 
access control. (Indeed, with 'module' accessibility for members, *only* 
the VM can possibly perform access control.)

To support these classloaders, the classfile must be in charge of which 
module it is a member of.

Finally, I would point out that if you package classes in a module 
archive, then purely at the conceptual level it seems reasonable for the 
classfiles to claim membership of that module. Under what conditions 
would the classfiles claim membership of another module?

> But we want to add compile-time access checks, and so, by definition, 
> the language must support them in some fashion. But what do we really 
> need here?
> 
> (You may recall a document I sent around prior to the official formation 
> of this JSR that described "module private" semantics; I'd re-send it 
> here for context but no longer have it due to a drive crash plus an IT 
> dept. backup disaster.)
> 
> For both compilation and runtime we need two functions:
> 
> 1. Membership: given a class and some artifact, determine its module.
> 2. Access: given a class, determine if it is accessible to a class in 
> another module.
> 
> Exports can then be defined using only these functions: the set of all 
> member classes which are accessible to other modules.
> 
> The access function seems obvious, and is inline with your suggestion 
> Andreas (and my original document): a 'module' qualifier which 
> translates to an access flag in the class file.
> 
> The membership function is clearly more interesting. At one extreme, we 
> could add a module name declaration to each source file; the class 
> itself becomes the artifact. At the other extreme we have a new 
> compilation unit and artifact, which lists all class members.
> 
> But both of these are brittle, and, just as bad, both ignore the version 
> problem. Given the value of module versions at runtime, shouldn't they 
> be just as important during compilation?
> 
> I argued in my original document that there is a happy medium between 
> these two extremes, one which supports versions during compilation 
> EXACTLY as will the runtime:
> 
> The compiler must USE the runtime.
> 
> That is, module membership should be defined by an abstraction, and the 
> compiler must use that abstraction. Given that pre-compiled dependencies 
> exist in the form of modules, the module system itself provides that 
> abstraction (ModuleDefinition) AND the access mechanism (Repository); 
> the compiler must merely use them.
> 
> Ok, so this makes sense once we know the dependencies, but... how do we 
> define these for the compiler? Today we give it a list of jars called a 
> classpath, so it seems to make sense that we could extend this to pass 
> in a list of module name/version pairs. But this is not as flexible as 
> the runtime, where version ranges and other constraints can be used to 
> select dependencies, nor does it allow the compiler to implement the 
> membership function for the source files.
> 
> So, back to something like a "super-package.java" file, but this time 
> lets support modules in a first class manner. Call it a "module.java" 
> file, and have it contain:
> 
> 1. A list of member *packages*, as in the current superpackage proposal.
> 2. A list of import declarations, exactly as required by 277.
> 
> The first enables the membership function; the second provides data for 
> the compiler to call Repository methods to find dependencies.
> 
> The module file can contain any additional annotations required by 277 
> (or any other module system). The binary form can then be used by a tool 
> to package up a .jam file (or an OSGi bundle with a bit more work).

I agree with the majority of the above (though an explicit export list 
is good for readability). I expect the 277 list will soon discuss how 
javac could build modules and check their dependencies on other modules.

Where I disagree is here: a Java compiler doesn't *have* to use modules. 
It cannot demand that every type it compiles is a member of a module or 
declares a dependency on a module or will be packaged in a module 
archive. The language does not demand such things. Ironically, the 
language does have to know about modules to handle legacy programs which 
unknowingly interact with them (as in my first scenario for migration 
compatibility) - but only to the degree of membership + access, not 
import dependencies and versioning and so on.

> At runtime, the module system binds each class to a Module by passing 
> the Module instance on the ClassLoader.defineClass() invocation.
> 
> This model does not require that the .class files contain module name 
> declarations, nor does it require a separate runtime binary solely for 
> the JVM's consumption. Either or both could certainly be added, if this 
> EG felt strongly that an extra level of membership enforcement is called 
> for. I believe that the module packaging is sufficient to define 
> membership, but others may disagree.
> 
> And it doesn't required nesting, the complications of which I don't 
> believe are worth the benefit.

I believe I have stated in this mail why I disagree that module 
packaging is sufficient to define membership. For now, I'm with you on 
nesting.

Alex