[jsr294-modularity-eg] Problems for JSR 294 to address

Mon Feb 2 20:06:23 EST 2009

I wonder why C# namespaces were not designed such that hierarchy governs 
accessibility?

Alex

Hal Hildebrand wrote:
> 
> On Feb 2, 2009, at 7:58 AM, Peter Kriens wrote:
> 
>> Hmm. .NET shows that you do not have to know all the module boundaries 
>> during compilation. 'internal' indicates inside the assembly and the 
>> source assembly is clearly not known during compilation. This concept 
>> looks attractively simple and is in line with common practice.
> 
> This is something I was wondering.  How can it be that .NET can do this, 
> but the gods of Java are unable to figure out how to accomplish this.
> 
>> For 294, during compilation, the JAR files and project source can 
>> easily be used inside the compiler for the scope. During runtime, the 
>> module system can set this boundary.
>>
>> However, I also like a module concept that is defined in the language. 
>> This will be more complex (and probably should be hierarchical) but it 
>> should be not attempt to define the deployment artifact nor 
>> dependencies. That is, it should be more like namespaces.
>>
>> Kind regards,
>>
>>     Peter Kriens
>>     
>> On 28 jan 2009, at 03:56, Alex Buckley wrote:
>>
>>> I am happy to discuss module-private accessibility first, in that it 
>>> addresses the limitations of packages on which we all agree.
>>>
>>> The issue we cannot ignore is that to check module-private 
>>> accessibility at compile-time means knowing the modules to which the 
>>> requesting and requested types belong. If I invoke:
>>>
>>> javac Foo.java Bar.java
>>>
>>> where Foo.java refers to Bar, then a compiler must be able to compute 
>>> at least the module name to which Foo belongs as Foo is compiled, if 
>>> Bar is found to be module-private as it is compiled. There are no 
>>> deployment artifacts around yet. It shouldn't be a surprise that 
>>> Foo.class and Bar.class are subsequently emitted to encode their 
>>> module membership.
>>>
>>> So with module accessibility comes module membership. Module 
>>> dependencies are a quite different topic which we'll get to soon 
>>> enough, and they are the place where minimizing new concepts is 
>>> essential.
>>>
>>> Alex
>>>
>>> Peter Kriens wrote:
>>>> I think it would make sense to work from first principles
>>>>  1. What problem are we trying to solve?
>>>>  2. How do we scope this problem to an acceptable area for all?
>>>>  3. What requirements should the solution fulfill
>>>> I would like to have a short discussion about the problem so we are 
>>>> sure we are all on the same page. I can start off with the problem I 
>>>> think we are trying to solve.
>>>> Modularity
>>>> Modularity is the art of encapsulation and hiding. A module limits 
>>>> the amount of information outside its boundaries. This reduces the 
>>>> overall complexity of a system because it becomes possible to reason 
>>>> (and change) locally about a module instead of understanding the 
>>>> whole system, and it provides a local namespace that is easier to 
>>>> work with for humans than a global namespace.
>>>> Java provides modularity in many places. A type encapsulates its 
>>>> fields and methods and selectively exposes them to other types with 
>>>> the access modifies: private, protected and public. A package 
>>>> encapsulates a number of types and resources. And last, but not 
>>>> least, the class loader provides a class space that can be distinct 
>>>> from other class spaces.
>>>> The modularity that is enabled with class loaders has been exploited 
>>>> by many. However, the Java Language has little to say about class 
>>>> loaders; it is outside the scope of the language.
>>>> So what problem do we need to solve, seeing that Java provides 
>>>> already so many forms of modularity? There is a sense in the 
>>>> community that packages are too small for modules. Though the 
>>>> package names hint at a hierarchy (and this is how they are found in 
>>>> a file system), this hierarchy does not provide a preferential 
>>>> treatment for children. Many programs consist of hundreds of 
>>>> packages. However, except for the limited package concept, the 
>>>> programmer cannot indicate that the visibility of an artifact should 
>>>> be limited to the program, or part of the program.
>>>> A large number of delivery formats have been created over the years, 
>>>> most of the based on the JAR format (which is also not a part of the 
>>>> language): WAR, EAR, OSGi bundles, midlets, xlets, and likely many 
>>>> more proprietary formats. Usually, the modularity of these modules 
>>>> is based on a rather simplistic hierarchical class loader model. 
>>>> That is, all classes in ancestors are visible, but not any siblings 
>>>> or their children. The language is moot on the point of this type of 
>>>> encapsulation.
>>>> Delivery modules contain types that depend on types in other 
>>>> delivery modules. There is no uniformly agreed Java standard to 
>>>> model these dependencies. During build time, the compiler is 
>>>> provided with a linear list of JARs and the compiler picks the first 
>>>> type that matches a name. Only the name is encoded in the class 
>>>> file, not the originating delivery module. During runtime, the same 
>>>> process is used to find classes, albeit in a hierarchical class 
>>>> loader model. This has all so far been outside the Java Language.
>>>> It is crucial that we distinguish the language/logic modularity and 
>>>> the modularity based on the deployment artifact. Interestingly, in 
>>>> my understanding, .NET makes such a distinction between the 
>>>> modularity of a delivery unit (called an assembly) and the finer 
>>>> grained modularity inside a delivery unit, represented by 
>>>> namespaces. The "internal" keyword indicates that the artifact is 
>>>> visible only inside an assembly and the "namespace" keyword provides 
>>>> a hierarchical namespace.
>>>> Therefore, one can identify two problems in the Java platform 
>>>> concerning modularity
>>>>  1. Packages are too limited for proper language/logic modularity
>>>>  2. Lack of a runtime module concept that is related to deployment
>>>>     artifacts (physical modularity)
>>>> Looking at the proposed time frame (EDR by mid-April), attacking 
>>>> both problems simultaneously seems rather unrealistic based on my 
>>>> limited experience. I therefore suggest to start with problem #1: 
>>>> Packages are too limited for proper language/logic modularity. What 
>>>> do you think?
>>>> Kind regards,
>>>> Peter Kriens
>>>> On 27 jan 2009, at 05:31, Alex Buckley wrote:
>>>>> Comments welcome.
>>>>>
>>>>> 1) Packages are typically arranged in hierarchies, but types can 
>>>>> only be
>>>>> used across different branches of the hierarchy by being made public,
>>>>> which exposes implementation details too widely. Information hiding is
>>>>> further reduced by interface members always being public.
>>>>>
>>>>> Non-solution: hierarchical package membership. Redefining existing
>>>>> well-known semantics is always a bad idea, and this one is especially
>>>>> complicated. There would need to be a way to stop package-private
>>>>> artifacts from being accessible to subpackages, or else we would be
>>>>> strengthening information hiding in one place only to weaken it
>>>>> elsewhere. Also, there would need to be a way to configure the 
>>>>> depth of
>>>>> exposure of package-private artifacts, since artifacts in package A
>>>>> should sometimes be accessible to A.B and not to A.B.C, and so on.
>>>>>
>>>>> 2) Dependencies of one type on another are expressed in source and
>>>>> classfiles, but there is no standard way for a Java compiler or JVM to
>>>>> interact with its environment to read and resolve dependencies. This
>>>>> causes compile-time and runtime environments to differ, and 
>>>>> complicates
>>>>> any effort to version types available in the (compile-time or runtime)
>>>>> environment.
>>>>>
>>>>> Non-solution: standardize the CLASSPATH. One list for all programs and
>>>>> libraries in the JVM makes versioning almost meaningless, and packages
>>>>> whose types occur in multiple CLASSPATH entries can behave poorly. 
>>>>> (See
>>>>> Peter Kriens' presentation at Devoxx 2008.)
>>>>>
>>>>> 3) Many packages have grown large over the years. They are effectively
>>>>> impossible to refactor now, since it is binary-incompatible to rename
>>>>> types and dangerous to "split" packages. The next-best option is to 
>>>>> let
>>>>> subsets be taken of existing packages, and deliver subsets
>>>>> independently, but there is no mechanism in the Java language or 
>>>>> JVM for
>>>>> that.
>>>>>
>>>>> Non-solution: a mechanism for renaming packages and types outside the
>>>>> Java language. This would require updating Java's model of binary
>>>>> compatibility, and would make source code incomprehensible.
>>>>>
>>>>> Alex
>>>>> _______________________________________________
>>>>> jsr294-modularity-eg mailing list
>>>>> jsr294-modularity-eg at cs.oswego.edu 
>>>>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> _______________________________________________
>>>> jsr294-modularity-eg mailing list
>>>> jsr294-modularity-eg at cs.oswego.edu
>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>> _______________________________________________
>>> jsr294-modularity-eg mailing list
>>> jsr294-modularity-eg at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
> 