[jsr294-modularity-eg] Problems for JSR 294 to address

[Hal Hildebrand]

On Feb 2, 2009, at 7:58 AM, Peter Kriens wrote:

> Hmm. .NET shows that you do not have to know all the module  
> boundaries during compilation. 'internal' indicates inside the  
> assembly and the source assembly is clearly not known during  
> compilation. This concept looks attractively simple and is in line  
> with common practice.

This is something I was wondering.  How can it be that .NET can do  
this, but the gods of Java are unable to figure out how to accomplish  
this.

> For 294, during compilation, the JAR files and project source can  
> easily be used inside the compiler for the scope. During runtime,  
> the module system can set this boundary.
>
> However, I also like a module concept that is defined in the  
> language. This will be more complex (and probably should be  
> hierarchical) but it should be not attempt to define the deployment  
> artifact nor dependencies. That is, it should be more like namespaces.
>
> Kind regards,
>
> 	Peter Kriens
> 	
> On 28 jan 2009, at 03:56, Alex Buckley wrote:
>
>> I am happy to discuss module-private accessibility first, in that  
>> it addresses the limitations of packages on which we all agree.
>>
>> The issue we cannot ignore is that to check module-private  
>> accessibility at compile-time means knowing the modules to which  
>> the requesting and requested types belong. If I invoke:
>>
>> javac Foo.java Bar.java
>>
>> where Foo.java refers to Bar, then a compiler must be able to  
>> compute at least the module name to which Foo belongs as Foo is  
>> compiled, if Bar is found to be module-private as it is compiled.  
>> There are no deployment artifacts around yet. It shouldn't be a  
>> surprise that Foo.class and Bar.class are subsequently emitted to  
>> encode their module membership.
>>
>> So with module accessibility comes module membership. Module  
>> dependencies are a quite different topic which we'll get to soon  
>> enough, and they are the place where minimizing new concepts is  
>> essential.
>>
>> Alex
>>
>> Peter Kriens wrote:
>>> I think it would make sense to work from first principles
>>>  1. What problem are we trying to solve?
>>>  2. How do we scope this problem to an acceptable area for all?
>>>  3. What requirements should the solution fulfill
>>> I would like to have a short discussion about the problem so we  
>>> are sure we are all on the same page. I can start off with the  
>>> problem I think we are trying to solve.
>>> Modularity
>>> Modularity is the art of encapsulation and hiding. A module limits  
>>> the amount of information outside its boundaries. This reduces the  
>>> overall complexity of a system because it becomes possible to  
>>> reason (and change) locally about a module instead of  
>>> understanding the whole system, and it provides a local namespace  
>>> that is easier to work with for humans than a global namespace.
>>> Java provides modularity in many places. A type encapsulates its  
>>> fields and methods and selectively exposes them to other types  
>>> with the access modifies: private, protected and public. A package  
>>> encapsulates a number of types and resources. And last, but not  
>>> least, the class loader provides a class space that can be  
>>> distinct from other class spaces.
>>> The modularity that is enabled with class loaders has been  
>>> exploited by many. However, the Java Language has little to say  
>>> about class loaders; it is outside the scope of the language.
>>> So what problem do we need to solve, seeing that Java provides  
>>> already so many forms of modularity? There is a sense in the  
>>> community that packages are too small for modules. Though the  
>>> package names hint at a hierarchy (and this is how they are found  
>>> in a file system), this hierarchy does not provide a preferential  
>>> treatment for children. Many programs consist of hundreds of  
>>> packages. However, except for the limited package concept, the  
>>> programmer cannot indicate that the visibility of an artifact  
>>> should be limited to the program, or part of the program.
>>> A large number of delivery formats have been created over the  
>>> years, most of the based on the JAR format (which is also not a  
>>> part of the language): WAR, EAR, OSGi bundles, midlets, xlets, and  
>>> likely many more proprietary formats. Usually, the modularity of  
>>> these modules is based on a rather simplistic hierarchical class  
>>> loader model. That is, all classes in ancestors are visible, but  
>>> not any siblings or their children. The language is moot on the  
>>> point of this type of encapsulation.
>>> Delivery modules contain types that depend on types in other  
>>> delivery modules. There is no uniformly agreed Java standard to  
>>> model these dependencies. During build time, the compiler is  
>>> provided with a linear list of JARs and the compiler picks the  
>>> first type that matches a name. Only the name is encoded in the  
>>> class file, not the originating delivery module. During runtime,  
>>> the same process is used to find classes, albeit in a hierarchical  
>>> class loader model. This has all so far been outside the Java  
>>> Language.
>>> It is crucial that we distinguish the language/logic modularity  
>>> and the modularity based on the deployment artifact.  
>>> Interestingly, in my understanding, .NET makes such a distinction  
>>> between the modularity of a delivery unit (called an assembly) and  
>>> the finer grained modularity inside a delivery unit, represented  
>>> by namespaces. The "internal" keyword indicates that the artifact  
>>> is visible only inside an assembly and the "namespace" keyword  
>>> provides a hierarchical namespace.
>>> Therefore, one can identify two problems in the Java platform  
>>> concerning modularity
>>>  1. Packages are too limited for proper language/logic modularity
>>>  2. Lack of a runtime module concept that is related to deployment
>>>     artifacts (physical modularity)
>>> Looking at the proposed time frame (EDR by mid-April), attacking  
>>> both problems simultaneously seems rather unrealistic based on my  
>>> limited experience. I therefore suggest to start with problem #1:  
>>> Packages are too limited for proper language/logic modularity.  
>>> What do you think?
>>> Kind regards,
>>> Peter Kriens
>>> On 27 jan 2009, at 05:31, Alex Buckley wrote:
>>>> Comments welcome.
>>>>
>>>> 1) Packages are typically arranged in hierarchies, but types can  
>>>> only be
>>>> used across different branches of the hierarchy by being made  
>>>> public,
>>>> which exposes implementation details too widely. Information  
>>>> hiding is
>>>> further reduced by interface members always being public.
>>>>
>>>> Non-solution: hierarchical package membership. Redefining existing
>>>> well-known semantics is always a bad idea, and this one is  
>>>> especially
>>>> complicated. There would need to be a way to stop package-private
>>>> artifacts from being accessible to subpackages, or else we would be
>>>> strengthening information hiding in one place only to weaken it
>>>> elsewhere. Also, there would need to be a way to configure the  
>>>> depth of
>>>> exposure of package-private artifacts, since artifacts in package A
>>>> should sometimes be accessible to A.B and not to A.B.C, and so on.
>>>>
>>>> 2) Dependencies of one type on another are expressed in source and
>>>> classfiles, but there is no standard way for a Java compiler or  
>>>> JVM to
>>>> interact with its environment to read and resolve dependencies.  
>>>> This
>>>> causes compile-time and runtime environments to differ, and  
>>>> complicates
>>>> any effort to version types available in the (compile-time or  
>>>> runtime)
>>>> environment.
>>>>
>>>> Non-solution: standardize the CLASSPATH. One list for all  
>>>> programs and
>>>> libraries in the JVM makes versioning almost meaningless, and  
>>>> packages
>>>> whose types occur in multiple CLASSPATH entries can behave  
>>>> poorly. (See
>>>> Peter Kriens' presentation at Devoxx 2008.)
>>>>
>>>> 3) Many packages have grown large over the years. They are  
>>>> effectively
>>>> impossible to refactor now, since it is binary-incompatible to  
>>>> rename
>>>> types and dangerous to "split" packages. The next-best option is  
>>>> to let
>>>> subsets be taken of existing packages, and deliver subsets
>>>> independently, but there is no mechanism in the Java language or  
>>>> JVM for
>>>> that.
>>>>
>>>> Non-solution: a mechanism for renaming packages and types outside  
>>>> the
>>>> Java language. This would require updating Java's model of binary
>>>> compatibility, and would make source code incomprehensible.
>>>>
>>>> Alex
>>>> _______________________________________________
>>>> jsr294-modularity-eg mailing list
>>>> jsr294-modularity-eg at cs.oswego.edu <mailto:jsr294-modularity-eg at cs.oswego.edu 
>>>> >
>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>> ------------------------------------------------------------------------
>>> _______________________________________________
>>> jsr294-modularity-eg mailing list
>>> jsr294-modularity-eg at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg