[jsr294-modularity-eg] Problems for JSR 294 to address

[Alex Buckley]

I am happy to discuss module-private accessibility first, in that it 
addresses the limitations of packages on which we all agree.

The issue we cannot ignore is that to check module-private accessibility 
at compile-time means knowing the modules to which the requesting and 
requested types belong. If I invoke:

javac Foo.java Bar.java

where Foo.java refers to Bar, then a compiler must be able to compute at 
least the module name to which Foo belongs as Foo is compiled, if Bar is 
found to be module-private as it is compiled. There are no deployment 
artifacts around yet. It shouldn't be a surprise that Foo.class and 
Bar.class are subsequently emitted to encode their module membership.

So with module accessibility comes module membership. Module 
dependencies are a quite different topic which we'll get to soon enough, 
and they are the place where minimizing new concepts is essential.

Alex

Peter Kriens wrote:
> I think it would make sense to work from first principles
> 
>    1. What problem are we trying to solve?
>    2. How do we scope this problem to an acceptable area for all?
>    3. What requirements should the solution fulfill
> 
> 
> I would like to have a short discussion about the problem so we are sure 
> we are all on the same page. I can start off with the problem I think we 
> are trying to solve.
> 
> Modularity
> Modularity is the art of encapsulation and hiding. A module limits the 
> amount of information outside its boundaries. This reduces the overall 
> complexity of a system because it becomes possible to reason (and 
> change) locally about a module instead of understanding the whole 
> system, and it provides a local namespace that is easier to work with 
> for humans than a global namespace.
> 
> Java provides modularity in many places. A type encapsulates its fields 
> and methods and selectively exposes them to other types with the access 
> modifies: private, protected and public. A package encapsulates a number 
> of types and resources. And last, but not least, the class loader 
> provides a class space that can be distinct from other class spaces.
> 
> The modularity that is enabled with class loaders has been exploited by 
> many. However, the Java Language has little to say about class loaders; 
> it is outside the scope of the language.
> 
> So what problem do we need to solve, seeing that Java provides already 
> so many forms of modularity? 
> 
> There is a sense in the community that packages are too small for 
> modules. Though the package names hint at a hierarchy (and this is how 
> they are found in a file system), this hierarchy does not provide a 
> preferential treatment for children. Many programs consist of hundreds 
> of packages. However, except for the limited package concept, the 
> programmer cannot indicate that the visibility of an artifact should be 
> limited to the program, or part of the program.
> 
> A large number of delivery formats have been created over the years, 
> most of the based on the JAR format (which is also not a part of the 
> language): WAR, EAR, OSGi bundles, midlets, xlets, and likely many more 
> proprietary formats. Usually, the modularity of these modules is based 
> on a rather simplistic hierarchical class loader model. That is, all 
> classes in ancestors are visible, but not any siblings or their 
> children. The language is moot on the point of this type of encapsulation.
> 
> Delivery modules contain types that depend on types in other delivery 
> modules. There is no uniformly agreed Java standard to model these 
> dependencies. During build time, the compiler is provided with a linear 
> list of JARs and the compiler picks the first type that matches a name. 
> Only the name is encoded in the class file, not the originating delivery 
> module. During runtime, the same process is used to find classes, albeit 
> in a hierarchical class loader model. This has all so far been outside 
> the Java Language.
> 
> It is crucial that we distinguish the language/logic modularity and the 
> modularity based on the deployment artifact. Interestingly, in my 
> understanding, .NET makes such a distinction between the modularity of a 
> delivery unit (called an assembly) and the finer grained modularity 
> inside a delivery unit, represented by namespaces. The "internal" 
> keyword indicates that the artifact is visible only inside an assembly 
> and the "namespace" keyword provides a hierarchical namespace.
> 
> Therefore, one can identify two problems in the Java platform concerning 
> modularity
> 
>    1. Packages are too limited for proper language/logic modularity
>    2. Lack of a runtime module concept that is related to deployment
>       artifacts (physical modularity)
> 
> 
> Looking at the proposed time frame (EDR by mid-April), attacking both 
> problems simultaneously seems rather unrealistic based on my limited 
> experience. I therefore suggest to start with problem #1: Packages are 
> too limited for proper language/logic modularity. 
> 
> What do you think?
> 
> Kind regards,
> 
> Peter Kriens
> 
> 
> On 27 jan 2009, at 05:31, Alex Buckley wrote:
> 
>> Comments welcome.
>>
>> 1) Packages are typically arranged in hierarchies, but types can only be
>> used across different branches of the hierarchy by being made public,
>> which exposes implementation details too widely. Information hiding is
>> further reduced by interface members always being public.
>>
>> Non-solution: hierarchical package membership. Redefining existing
>> well-known semantics is always a bad idea, and this one is especially
>> complicated. There would need to be a way to stop package-private
>> artifacts from being accessible to subpackages, or else we would be
>> strengthening information hiding in one place only to weaken it
>> elsewhere. Also, there would need to be a way to configure the depth of
>> exposure of package-private artifacts, since artifacts in package A
>> should sometimes be accessible to A.B and not to A.B.C, and so on.
>>
>> 2) Dependencies of one type on another are expressed in source and
>> classfiles, but there is no standard way for a Java compiler or JVM to
>> interact with its environment to read and resolve dependencies. This
>> causes compile-time and runtime environments to differ, and complicates
>> any effort to version types available in the (compile-time or runtime)
>> environment.
>>
>> Non-solution: standardize the CLASSPATH. One list for all programs and
>> libraries in the JVM makes versioning almost meaningless, and packages
>> whose types occur in multiple CLASSPATH entries can behave poorly. (See
>> Peter Kriens' presentation at Devoxx 2008.)
>>
>> 3) Many packages have grown large over the years. They are effectively
>> impossible to refactor now, since it is binary-incompatible to rename
>> types and dangerous to "split" packages. The next-best option is to let
>> subsets be taken of existing packages, and deliver subsets
>> independently, but there is no mechanism in the Java language or JVM for
>> that.
>>
>> Non-solution: a mechanism for renaming packages and types outside the
>> Java language. This would require updating Java's model of binary
>> compatibility, and would make source code incomprehensible.
>>
>> Alex
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu 
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg