[jsr294-modularity-eg] Module boundaries
Peter Kriens
peter.kriens at aqute.biz
Fri Feb 13 13:53:48 EST 2009
Enclosed find the first strawman proposal for nested modules. This was
not easy :-( but I tried hard to make it as clear as possible. I guess
sending docs will probably not work or create too much CO2, so here is
the link: http://www.aqute.biz/JSR294/HomePage. Enjoy.
Kind regards,
Peter Kriens
On 4 feb 2009, at 10:32, Alex Buckley wrote:
> Yes, setting module membership at runtime effectively simulates
> 'internal', where the .NET assembly-build operation is done lazily.
>
> Module nesting sounds interesting. Could you please send some source
> code which exhibits how it would impact module-private accessibility?
>
> We can certainly have a phone call. I am travelling internationally
> for the next week and, in practice, will not be back in the office
> until Tuesday 17th.
>
> EG members, please indicate when on 17-20th February you could be
> available. Good times would be 9am West Coast, 12pm East Coast, 6pm
> France.
>
> Alex
>
> Peter Kriens wrote:
>> Ok, so you allow a module boundary to be set in runtime, this will
>> make the module keyword in unbound modules reflect the semantics of
>> the .NET 'internal' concept? I agree we should then have a
>> discussion later about how the compiler gets the unbound module
>> boundaries.(Do we have an action list for these things?)
>> If I look at software projects than there will be a tendency to put
>> a set of related packages into a bound module, where a number of
>> these bound modules are packaged in a JAR (the unbound module).
>> This seems to scream for nesting of bound modules?
>> VM -> unbound module -> bound module +-> package -> type -+
>> ^ | ^ |
>> +--------------+ +--------+
>> And, can we please, pretty please, have a kick-off phone conference
>> now we're a group starting to work on this?
>> Kind regards,
>> Peter Kriens
>> On 3 feb 2009, at 01:47, Alex Buckley wrote:
>>> .NET is not Java.
>>>
>>> Compiled types in .NET live in non-executable modules (.netmodule
>>> files). To be executed, modules must be packaged into assemblies
>>> (.dll/.exe files).
>>> This packaging makes "internal" a bit tricky. "internal" is
>>> specified as "accessible from the same program", where a program
>>> is merely a bunch of source files. A C# compiler must take it on
>>> trust that the .netmodule which accesses an "internal" artifact
>>> will be packaged in the same assembly as the .netmodule which
>>> declares the artifact.
>>>
>>> This is not unreasonable, since .netmodules MUST be packaged. But
>>> if packaged "wrongly", code accessing an "internal" artifact which
>>> the compiler promised was accessible will fail at runtime. You do
>>> not want language features to behave like this. There is
>>> effectively a hole in the C# spec caused by the greater CLI spec
>>> stratifying loadable code into non-executable and executable.
>>>
>>> Java is different. A JVM does not require classfiles to be
>>> packaged in any way before executing them. Nor does the reference
>>> implementation of a Java compiler require classfiles to be
>>> packaged before compiling against them. These things are not going
>>> to change. Since module-private accessibility is a language and VM
>>> construct, it cannot depend on the future packaging of code still
>>> being compiled or compiled against.
>>>
>>> (What it can depend on is an API consulted by a Java compiler to
>>> discover whether module M version X and module M version Y have
>>> "equal" versions. The API implementation would be honor-bound to
>>> give consistent answers, similar to classloaders. We'll get to
>>> that API soon, I hope.)
>>>
>>> This boils down to saying the module-private accessibility and
>>> module membership go hand-in-hand for me. In any case, the first
>>> thing a programmer will ask when seeing a module-private member
>>> is: "What module does this member belong to?". The answer should
>>> NOT be "it depends on a compiler flag or invocation of jar".
>>>
>>> I will shortly send a Java language grammar and classfile format
>>> which reflect these views.
>>>
>>> Now, I know this issue is not going to go away. I therefore
>>> propose that IN ADDITION to encoding module membership in source,
>>> it should be possible for the host system to determine module
>>> membership. If you REALLY want the answer above to be "it depends
>>> on a compiler flag or invocation of jar", you can do that. But a
>>> Java compiler/VM must defer to the membership in a source/class
>>> file if it exists.
>>>
>>> Is that enough of a weakening of my requirements #1 and #2?
>>>
>>> Alex
>>>
>>> Peter Kriens wrote:
>>>> Hmm. .NET shows that you do not have to know all the module
>>>> boundaries during compilation. 'internal' indicates inside the
>>>> assembly and the source assembly is clearly not known during
>>>> compilation. This concept looks attractively simple and is in
>>>> line with common practice.
>>>> For 294, during compilation, the JAR files and project source can
>>>> easily be used inside the compiler for the scope. During runtime,
>>>> the module system can set this boundary.
>>>> However, I also like a module concept that is defined in the
>>>> language. This will be more complex (and probably should be
>>>> hierarchical) but it should be not attempt to define the
>>>> deployment artifact nor dependencies. That is, it should be more
>>>> like namespaces.
>>>> Kind regards,
>>>> Peter Kriens
>>>> On 28 jan 2009, at 03:56, Alex Buckley wrote:
>>>>> I am happy to discuss module-private accessibility first, in
>>>>> that it addresses the limitations of packages on which we all
>>>>> agree.
>>>>>
>>>>> The issue we cannot ignore is that to check module-private
>>>>> accessibility at compile-time means knowing the modules to which
>>>>> the requesting and requested types belong. If I invoke:
>>>>>
>>>>> javac Foo.java Bar.java
>>>>>
>>>>> where Foo.java refers to Bar, then a compiler must be able to
>>>>> compute at least the module name to which Foo belongs as Foo is
>>>>> compiled, if Bar is found to be module-private as it is
>>>>> compiled. There are no deployment artifacts around yet. It
>>>>> shouldn't be a surprise that Foo.class and Bar.class are
>>>>> subsequently emitted to encode their module membership.
>>>>>
>>>>> So with module accessibility comes module membership. Module
>>>>> dependencies are a quite different topic which we'll get to soon
>>>>> enough, and they are the place where minimizing new concepts is
>>>>> essential.
>>>>>
>>>>> Alex
>>>>>
>>>>> Peter Kriens wrote:
>>>>>> I think it would make sense to work from first principles
>>>>>> 1. What problem are we trying to solve?
>>>>>> 2. How do we scope this problem to an acceptable area for all?
>>>>>> 3. What requirements should the solution fulfill
>>>>>> I would like to have a short discussion about the problem so we
>>>>>> are sure we are all on the same page. I can start off with the
>>>>>> problem I think we are trying to solve.
>>>>>> Modularity
>>>>>> Modularity is the art of encapsulation and hiding. A module
>>>>>> limits the amount of information outside its boundaries. This
>>>>>> reduces the overall complexity of a system because it becomes
>>>>>> possible to reason (and change) locally about a module instead
>>>>>> of understanding the whole system, and it provides a local
>>>>>> namespace that is easier to work with for humans than a global
>>>>>> namespace.
>>>>>> Java provides modularity in many places. A type encapsulates
>>>>>> its fields and methods and selectively exposes them to other
>>>>>> types with the access modifies: private, protected and public.
>>>>>> A package encapsulates a number of types and resources. And
>>>>>> last, but not least, the class loader provides a class space
>>>>>> that can be distinct from other class spaces.
>>>>>> The modularity that is enabled with class loaders has been
>>>>>> exploited by many. However, the Java Language has little to say
>>>>>> about class loaders; it is outside the scope of the language.
>>>>>> So what problem do we need to solve, seeing that Java provides
>>>>>> already so many forms of modularity? There is a sense in the
>>>>>> community that packages are too small for modules. Though the
>>>>>> package names hint at a hierarchy (and this is how they are
>>>>>> found in a file system), this hierarchy does not provide a
>>>>>> preferential treatment for children. Many programs consist of
>>>>>> hundreds of packages. However, except for the limited package
>>>>>> concept, the programmer cannot indicate that the visibility of
>>>>>> an artifact should be limited to the program, or part of the
>>>>>> program.
>>>>>> A large number of delivery formats have been created over the
>>>>>> years, most of the based on the JAR format (which is also not a
>>>>>> part of the language): WAR, EAR, OSGi bundles, midlets, xlets,
>>>>>> and likely many more proprietary formats. Usually, the
>>>>>> modularity of these modules is based on a rather simplistic
>>>>>> hierarchical class loader model. That is, all classes in
>>>>>> ancestors are visible, but not any siblings or their children.
>>>>>> The language is moot on the point of this type of encapsulation.
>>>>>> Delivery modules contain types that depend on types in other
>>>>>> delivery modules. There is no uniformly agreed Java standard to
>>>>>> model these dependencies. During build time, the compiler is
>>>>>> provided with a linear list of JARs and the compiler picks the
>>>>>> first type that matches a name. Only the name is encoded in the
>>>>>> class file, not the originating delivery module. During
>>>>>> runtime, the same process is used to find classes, albeit in a
>>>>>> hierarchical class loader model. This has all so far been
>>>>>> outside the Java Language.
>>>>>> It is crucial that we distinguish the language/logic modularity
>>>>>> and the modularity based on the deployment artifact.
>>>>>> Interestingly, in my understanding, .NET makes such a
>>>>>> distinction between the modularity of a delivery unit (called
>>>>>> an assembly) and the finer grained modularity inside a delivery
>>>>>> unit, represented by namespaces. The "internal" keyword
>>>>>> indicates that the artifact is visible only inside an assembly
>>>>>> and the "namespace" keyword provides a hierarchical namespace.
>>>>>> Therefore, one can identify two problems in the Java platform
>>>>>> concerning modularity
>>>>>> 1. Packages are too limited for proper language/logic modularity
>>>>>> 2. Lack of a runtime module concept that is related to deployment
>>>>>> artifacts (physical modularity)
>>>>>> Looking at the proposed time frame (EDR by mid-April),
>>>>>> attacking both problems simultaneously seems rather unrealistic
>>>>>> based on my limited experience. I therefore suggest to start
>>>>>> with problem #1: Packages are too limited for proper language/
>>>>>> logic modularity. What do you think?
>>>>>> Kind regards,
>>>>>> Peter Kriens
>>>>>> On 27 jan 2009, at 05:31, Alex Buckley wrote:
>>>>>>> Comments welcome.
>>>>>>>
>>>>>>> 1) Packages are typically arranged in hierarchies, but types
>>>>>>> can only be
>>>>>>> used across different branches of the hierarchy by being made
>>>>>>> public,
>>>>>>> which exposes implementation details too widely. Information
>>>>>>> hiding is
>>>>>>> further reduced by interface members always being public.
>>>>>>>
>>>>>>> Non-solution: hierarchical package membership. Redefining
>>>>>>> existing
>>>>>>> well-known semantics is always a bad idea, and this one is
>>>>>>> especially
>>>>>>> complicated. There would need to be a way to stop package-
>>>>>>> private
>>>>>>> artifacts from being accessible to subpackages, or else we
>>>>>>> would be
>>>>>>> strengthening information hiding in one place only to weaken it
>>>>>>> elsewhere. Also, there would need to be a way to configure the
>>>>>>> depth of
>>>>>>> exposure of package-private artifacts, since artifacts in
>>>>>>> package A
>>>>>>> should sometimes be accessible to A.B and not to A.B.C, and so
>>>>>>> on.
>>>>>>>
>>>>>>> 2) Dependencies of one type on another are expressed in source
>>>>>>> and
>>>>>>> classfiles, but there is no standard way for a Java compiler
>>>>>>> or JVM to
>>>>>>> interact with its environment to read and resolve
>>>>>>> dependencies. This
>>>>>>> causes compile-time and runtime environments to differ, and
>>>>>>> complicates
>>>>>>> any effort to version types available in the (compile-time or
>>>>>>> runtime)
>>>>>>> environment.
>>>>>>>
>>>>>>> Non-solution: standardize the CLASSPATH. One list for all
>>>>>>> programs and
>>>>>>> libraries in the JVM makes versioning almost meaningless, and
>>>>>>> packages
>>>>>>> whose types occur in multiple CLASSPATH entries can behave
>>>>>>> poorly. (See
>>>>>>> Peter Kriens' presentation at Devoxx 2008.)
>>>>>>>
>>>>>>> 3) Many packages have grown large over the years. They are
>>>>>>> effectively
>>>>>>> impossible to refactor now, since it is binary-incompatible to
>>>>>>> rename
>>>>>>> types and dangerous to "split" packages. The next-best option
>>>>>>> is to let
>>>>>>> subsets be taken of existing packages, and deliver subsets
>>>>>>> independently, but there is no mechanism in the Java language
>>>>>>> or JVM for
>>>>>>> that.
>>>>>>>
>>>>>>> Non-solution: a mechanism for renaming packages and types
>>>>>>> outside the
>>>>>>> Java language. This would require updating Java's model of
>>>>>>> binary
>>>>>>> compatibility, and would make source code incomprehensible.
>>>>>>>
>>>>>>> Alex
>>>>>>> _______________________________________________
>>>>>>> jsr294-modularity-eg mailing list
>>>>>>> jsr294-modularity-eg at cs.oswego.edu <mailto:jsr294-modularity-eg at cs.oswego.edu
>>>>>>> >
>>>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>>> ------------------------------------------------------------------------
>>>>>> _______________________________________________
>>>>>> jsr294-modularity-eg mailing list
>>>>>> jsr294-modularity-eg at cs.oswego.edu
>>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>> _______________________________________________
>>>>> jsr294-modularity-eg mailing list
>>>>> jsr294-modularity-eg at cs.oswego.edu
>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>> _______________________________________________
>>>> jsr294-modularity-eg mailing list
>>>> jsr294-modularity-eg at cs.oswego.edu
>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>> _______________________________________________
>>> jsr294-modularity-eg mailing list
>>> jsr294-modularity-eg at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
More information about the jsr294-modularity-eg
mailing list