[jsr294-modularity-eg] Module boundaries

Wed Feb 4 03:02:45 EST 2009

Ok, so you allow a module boundary to be set in runtime, this will  
make the module keyword in unbound modules reflect the semantics of  
the .NET 'internal' concept? I agree we should then have a discussion  
later about how the compiler gets the unbound module boundaries.(Do we  
have an action list for these things?)

If I look at software projects than there will be a tendency to put a  
set of related packages into a bound module, where a number of these  
bound modules are packaged in a JAR (the unbound module). This seems  
to scream for nesting of bound modules?

   VM -> unbound module -> bound module +-> package -> type -+
                         ^              |           ^        |
                         +--------------+           +--------+

And, can we please, pretty please, have a kick-off phone conference  
now we're a group starting to work on this?

Kind regards,

	Peter Kriens

On 3 feb 2009, at 01:47, Alex Buckley wrote:

> .NET is not Java.
>
> Compiled types in .NET live in non-executable modules (.netmodule  
> files). To be executed, modules must be packaged into assemblies  
> (.dll/.exe files).
> This packaging makes "internal" a bit tricky. "internal" is  
> specified as "accessible from the same program", where a program is  
> merely a bunch of source files. A C# compiler must take it on trust  
> that the .netmodule which accesses an "internal" artifact will be  
> packaged in the same assembly as the .netmodule which declares the  
> artifact.
>
> This is not unreasonable, since .netmodules MUST be packaged. But if  
> packaged "wrongly", code accessing an "internal" artifact which the  
> compiler promised was accessible will fail at runtime. You do not  
> want language features to behave like this. There is effectively a  
> hole in the C# spec caused by the greater CLI spec stratifying  
> loadable code into non-executable and executable.
>
> Java is different. A JVM does not require classfiles to be packaged  
> in any way before executing them. Nor does the reference  
> implementation of a Java compiler require classfiles to be packaged  
> before compiling against them. These things are not going to change.  
> Since module-private accessibility is a language and VM construct,  
> it cannot depend on the future packaging of code still being  
> compiled or compiled against.
>
> (What it can depend on is an API consulted by a Java compiler to  
> discover whether module M version X and module M version Y have  
> "equal" versions. The API implementation would be honor-bound to  
> give consistent answers, similar to classloaders. We'll get to that  
> API soon, I hope.)
>
> This boils down to saying the module-private accessibility and  
> module membership go hand-in-hand for me. In any case, the first  
> thing a programmer will ask when seeing a module-private member is:  
> "What module does this member belong to?". The answer should NOT be  
> "it depends on a compiler flag or invocation of jar".
>
> I will shortly send a Java language grammar and classfile format  
> which reflect these views.
>
> Now, I know this issue is not going to go away. I therefore propose  
> that IN ADDITION to encoding module membership in source, it should  
> be possible for the host system to determine module membership. If  
> you REALLY want the answer above to be "it depends on a compiler  
> flag or invocation of jar", you can do that. But a Java compiler/VM  
> must defer to the membership in a source/class file if it exists.
>
> Is that enough of a weakening of my requirements #1 and #2?
>
> Alex
>
> Peter Kriens wrote:
>> Hmm. .NET shows that you do not have to know all the module  
>> boundaries during compilation. 'internal' indicates inside the  
>> assembly and the source assembly is clearly not known during  
>> compilation. This concept looks attractively simple and is in line  
>> with common practice.
>> For 294, during compilation, the JAR files and project source can  
>> easily be used inside the compiler for the scope. During runtime,  
>> the module system can set this boundary.
>> However, I also like a module concept that is defined in the  
>> language. This will be more complex (and probably should be  
>> hierarchical) but it should be not attempt to define the deployment  
>> artifact nor dependencies. That is, it should be more like  
>> namespaces.
>> Kind regards,
>>    Peter Kriens
>>    On 28 jan 2009, at 03:56, Alex Buckley wrote:
>>> I am happy to discuss module-private accessibility first, in that  
>>> it addresses the limitations of packages on which we all agree.
>>>
>>> The issue we cannot ignore is that to check module-private  
>>> accessibility at compile-time means knowing the modules to which  
>>> the requesting and requested types belong. If I invoke:
>>>
>>> javac Foo.java Bar.java
>>>
>>> where Foo.java refers to Bar, then a compiler must be able to  
>>> compute at least the module name to which Foo belongs as Foo is  
>>> compiled, if Bar is found to be module-private as it is compiled.  
>>> There are no deployment artifacts around yet. It shouldn't be a  
>>> surprise that Foo.class and Bar.class are subsequently emitted to  
>>> encode their module membership.
>>>
>>> So with module accessibility comes module membership. Module  
>>> dependencies are a quite different topic which we'll get to soon  
>>> enough, and they are the place where minimizing new concepts is  
>>> essential.
>>>
>>> Alex
>>>
>>> Peter Kriens wrote:
>>>> I think it would make sense to work from first principles
>>>>  1. What problem are we trying to solve?
>>>>  2. How do we scope this problem to an acceptable area for all?
>>>>  3. What requirements should the solution fulfill
>>>> I would like to have a short discussion about the problem so we  
>>>> are sure we are all on the same page. I can start off with the  
>>>> problem I think we are trying to solve.
>>>> Modularity
>>>> Modularity is the art of encapsulation and hiding. A module  
>>>> limits the amount of information outside its boundaries. This  
>>>> reduces the overall complexity of a system because it becomes  
>>>> possible to reason (and change) locally about a module instead of  
>>>> understanding the whole system, and it provides a local namespace  
>>>> that is easier to work with for humans than a global namespace.
>>>> Java provides modularity in many places. A type encapsulates its  
>>>> fields and methods and selectively exposes them to other types  
>>>> with the access modifies: private, protected and public. A  
>>>> package encapsulates a number of types and resources. And last,  
>>>> but not least, the class loader provides a class space that can  
>>>> be distinct from other class spaces.
>>>> The modularity that is enabled with class loaders has been  
>>>> exploited by many. However, the Java Language has little to say  
>>>> about class loaders; it is outside the scope of the language.
>>>> So what problem do we need to solve, seeing that Java provides  
>>>> already so many forms of modularity? There is a sense in the  
>>>> community that packages are too small for modules. Though the  
>>>> package names hint at a hierarchy (and this is how they are found  
>>>> in a file system), this hierarchy does not provide a preferential  
>>>> treatment for children. Many programs consist of hundreds of  
>>>> packages. However, except for the limited package concept, the  
>>>> programmer cannot indicate that the visibility of an artifact  
>>>> should be limited to the program, or part of the program.
>>>> A large number of delivery formats have been created over the  
>>>> years, most of the based on the JAR format (which is also not a  
>>>> part of the language): WAR, EAR, OSGi bundles, midlets, xlets,  
>>>> and likely many more proprietary formats. Usually, the modularity  
>>>> of these modules is based on a rather simplistic hierarchical  
>>>> class loader model. That is, all classes in ancestors are  
>>>> visible, but not any siblings or their children. The language is  
>>>> moot on the point of this type of encapsulation.
>>>> Delivery modules contain types that depend on types in other  
>>>> delivery modules. There is no uniformly agreed Java standard to  
>>>> model these dependencies. During build time, the compiler is  
>>>> provided with a linear list of JARs and the compiler picks the  
>>>> first type that matches a name. Only the name is encoded in the  
>>>> class file, not the originating delivery module. During runtime,  
>>>> the same process is used to find classes, albeit in a  
>>>> hierarchical class loader model. This has all so far been outside  
>>>> the Java Language.
>>>> It is crucial that we distinguish the language/logic modularity  
>>>> and the modularity based on the deployment artifact.  
>>>> Interestingly, in my understanding, .NET makes such a distinction  
>>>> between the modularity of a delivery unit (called an assembly)  
>>>> and the finer grained modularity inside a delivery unit,  
>>>> represented by namespaces. The "internal" keyword indicates that  
>>>> the artifact is visible only inside an assembly and the  
>>>> "namespace" keyword provides a hierarchical namespace.
>>>> Therefore, one can identify two problems in the Java platform  
>>>> concerning modularity
>>>>  1. Packages are too limited for proper language/logic modularity
>>>>  2. Lack of a runtime module concept that is related to deployment
>>>>     artifacts (physical modularity)
>>>> Looking at the proposed time frame (EDR by mid-April), attacking  
>>>> both problems simultaneously seems rather unrealistic based on my  
>>>> limited experience. I therefore suggest to start with problem #1:  
>>>> Packages are too limited for proper language/logic modularity.  
>>>> What do you think?
>>>> Kind regards,
>>>> Peter Kriens
>>>> On 27 jan 2009, at 05:31, Alex Buckley wrote:
>>>>> Comments welcome.
>>>>>
>>>>> 1) Packages are typically arranged in hierarchies, but types can  
>>>>> only be
>>>>> used across different branches of the hierarchy by being made  
>>>>> public,
>>>>> which exposes implementation details too widely. Information  
>>>>> hiding is
>>>>> further reduced by interface members always being public.
>>>>>
>>>>> Non-solution: hierarchical package membership. Redefining existing
>>>>> well-known semantics is always a bad idea, and this one is  
>>>>> especially
>>>>> complicated. There would need to be a way to stop package-private
>>>>> artifacts from being accessible to subpackages, or else we would  
>>>>> be
>>>>> strengthening information hiding in one place only to weaken it
>>>>> elsewhere. Also, there would need to be a way to configure the  
>>>>> depth of
>>>>> exposure of package-private artifacts, since artifacts in  
>>>>> package A
>>>>> should sometimes be accessible to A.B and not to A.B.C, and so on.
>>>>>
>>>>> 2) Dependencies of one type on another are expressed in source and
>>>>> classfiles, but there is no standard way for a Java compiler or  
>>>>> JVM to
>>>>> interact with its environment to read and resolve dependencies.  
>>>>> This
>>>>> causes compile-time and runtime environments to differ, and  
>>>>> complicates
>>>>> any effort to version types available in the (compile-time or  
>>>>> runtime)
>>>>> environment.
>>>>>
>>>>> Non-solution: standardize the CLASSPATH. One list for all  
>>>>> programs and
>>>>> libraries in the JVM makes versioning almost meaningless, and  
>>>>> packages
>>>>> whose types occur in multiple CLASSPATH entries can behave  
>>>>> poorly. (See
>>>>> Peter Kriens' presentation at Devoxx 2008.)
>>>>>
>>>>> 3) Many packages have grown large over the years. They are  
>>>>> effectively
>>>>> impossible to refactor now, since it is binary-incompatible to  
>>>>> rename
>>>>> types and dangerous to "split" packages. The next-best option is  
>>>>> to let
>>>>> subsets be taken of existing packages, and deliver subsets
>>>>> independently, but there is no mechanism in the Java language or  
>>>>> JVM for
>>>>> that.
>>>>>
>>>>> Non-solution: a mechanism for renaming packages and types  
>>>>> outside the
>>>>> Java language. This would require updating Java's model of binary
>>>>> compatibility, and would make source code incomprehensible.
>>>>>
>>>>> Alex
>>>>> _______________________________________________
>>>>> jsr294-modularity-eg mailing list
>>>>> jsr294-modularity-eg at cs.oswego.edu <mailto:jsr294-modularity-eg at cs.oswego.edu 
>>>>> >
>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>> ------------------------------------------------------------------------
>>>> _______________________________________________
>>>> jsr294-modularity-eg mailing list
>>>> jsr294-modularity-eg at cs.oswego.edu
>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>> _______________________________________________
>>> jsr294-modularity-eg mailing list
>>> jsr294-modularity-eg at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg

_______________________________________________
jsr294-modularity-eg mailing list
jsr294-modularity-eg at cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg