[jsr294-modularity-eg] Module boundaries

Christopher Brind brindy at brindy.org.uk
Fri Feb 13 16:45:44 EST 2009 

Hi,

I have to admit that I read the straw-man with a mixture of both excitement
and trepidation. :)

The first question that springs to mind is, what is to stop me from
contributing to the 'Felix' module example in a way that then exposes some
of the module private functionality, e.g. via a delegate?

So, for example, let's say that I have downloaded a JAR which contains the
Felix classes from the example in the straw-man.  I could compile the
following in order to contribute to the Felix module (source tree example
like in the straw-man):
module-info: module Felix;

org/apache/felix/framework
    PublicFelix: package org.apache.felix.framework;
              public class PublicFelix {  ... }

I then repackage the downloaded JAR adding the PublicFelix class.  The
result is that because I told the compiler that the PublicFelix class is in
the Felix module, it can access the Felix class.  The code in the
PublicFelix class is basically a delegate for the Felix class, but since the
PublicFelix class is accessible from anywhere the ultimate result is that
the module system has been circumvented.

The implication of this seems to be that if you want to prevent this from
happening then you will have to provide the binaries in a format which
prevents tampering.  For instance, if the original Felix module was compiled
and then bundled as a signed JAR I could not contribute to the module as
described above as that will break the JAR's signature.  However, the
proposal says that types in a module can be loaded from multiple containers,
e.g. URL Class Loader.  How do I protect my module's internal implementation
in this case?

Regards,
Chris



2009/2/13 Peter Kriens <peter.kriens at aqute.biz>

> Enclosed find the first strawman proposal for nested modules. This was not
> easy :-( but I tried hard to make it as clear as possible. I guess sending
> docs will probably not work or create too much CO2, so here is the link:
> http://www.aqute.biz/JSR294/HomePage. Enjoy.
>
> Kind regards,
>
>        Peter Kriens
>
>
> On 4 feb 2009, at 10:32, Alex Buckley wrote:
>
>  Yes, setting module membership at runtime effectively simulates
>> 'internal', where the .NET assembly-build operation is done lazily.
>>
>> Module nesting sounds interesting. Could you please send some source code
>> which exhibits how it would impact module-private accessibility?
>>
>> We can certainly have a phone call. I am travelling internationally for
>> the next week and, in practice, will not be back in the office until Tuesday
>> 17th.
>>
>> EG members, please indicate when on 17-20th February you could be
>> available. Good times would be 9am West Coast, 12pm East Coast, 6pm France.
>>
>> Alex
>>
>> Peter Kriens wrote:
>>
>>> Ok, so you allow a module boundary to be set in runtime, this will make
>>> the module keyword in unbound modules reflect the semantics of the .NET
>>> 'internal' concept? I agree we should then have a discussion later about how
>>> the compiler gets the unbound module boundaries.(Do we have an action list
>>> for these things?)
>>> If I look at software projects than there will be a tendency to put a set
>>> of related packages into a bound module, where a number of these bound
>>> modules are packaged in a JAR (the unbound module). This seems to scream for
>>> nesting of bound modules?
>>>  VM -> unbound module -> bound module +-> package -> type -+
>>>                       ^              |           ^        |
>>>                       +--------------+           +--------+
>>> And, can we please, pretty please, have a kick-off phone conference now
>>> we're a group starting to work on this?
>>> Kind regards,
>>>   Peter Kriens
>>> On 3 feb 2009, at 01:47, Alex Buckley wrote:
>>>
>>>> .NET is not Java.
>>>>
>>>> Compiled types in .NET live in non-executable modules (.netmodule
>>>> files). To be executed, modules must be packaged into assemblies (.dll/.exe
>>>> files).
>>>> This packaging makes "internal" a bit tricky. "internal" is specified as
>>>> "accessible from the same program", where a program is merely a bunch of
>>>> source files. A C# compiler must take it on trust that the .netmodule which
>>>> accesses an "internal" artifact will be packaged in the same assembly as the
>>>> .netmodule which declares the artifact.
>>>>
>>>> This is not unreasonable, since .netmodules MUST be packaged. But if
>>>> packaged "wrongly", code accessing an "internal" artifact which the compiler
>>>> promised was accessible will fail at runtime. You do not want language
>>>> features to behave like this. There is effectively a hole in the C# spec
>>>> caused by the greater CLI spec stratifying loadable code into non-executable
>>>> and executable.
>>>>
>>>> Java is different. A JVM does not require classfiles to be packaged in
>>>> any way before executing them. Nor does the reference implementation of a
>>>> Java compiler require classfiles to be packaged before compiling against
>>>> them. These things are not going to change. Since module-private
>>>> accessibility is a language and VM construct, it cannot depend on the future
>>>> packaging of code still being compiled or compiled against.
>>>>
>>>> (What it can depend on is an API consulted by a Java compiler to
>>>> discover whether module M version X and module M version Y have "equal"
>>>> versions. The API implementation would be honor-bound to give consistent
>>>> answers, similar to classloaders. We'll get to that API soon, I hope.)
>>>>
>>>> This boils down to saying the module-private accessibility and module
>>>> membership go hand-in-hand for me. In any case, the first thing a programmer
>>>> will ask when seeing a module-private member is: "What module does this
>>>> member belong to?". The answer should NOT be "it depends on a compiler flag
>>>> or invocation of jar".
>>>>
>>>> I will shortly send a Java language grammar and classfile format which
>>>> reflect these views.
>>>>
>>>> Now, I know this issue is not going to go away. I therefore propose that
>>>> IN ADDITION to encoding module membership in source, it should be possible
>>>> for the host system to determine module membership. If you REALLY want the
>>>> answer above to be "it depends on a compiler flag or invocation of jar", you
>>>> can do that. But a Java compiler/VM must defer to the membership in a
>>>> source/class file if it exists.
>>>>
>>>> Is that enough of a weakening of my requirements #1 and #2?
>>>>
>>>> Alex
>>>>
>>>> Peter Kriens wrote:
>>>>
>>>>> Hmm. .NET shows that you do not have to know all the module boundaries
>>>>> during compilation. 'internal' indicates inside the assembly and the source
>>>>> assembly is clearly not known during compilation. This concept looks
>>>>> attractively simple and is in line with common practice.
>>>>> For 294, during compilation, the JAR files and project source can
>>>>> easily be used inside the compiler for the scope. During runtime, the module
>>>>> system can set this boundary.
>>>>> However, I also like a module concept that is defined in the language.
>>>>> This will be more complex (and probably should be hierarchical) but it
>>>>> should be not attempt to define the deployment artifact nor dependencies.
>>>>> That is, it should be more like namespaces.
>>>>> Kind regards,
>>>>>  Peter Kriens
>>>>>  On 28 jan 2009, at 03:56, Alex Buckley wrote:
>>>>>
>>>>>> I am happy to discuss module-private accessibility first, in that it
>>>>>> addresses the limitations of packages on which we all agree.
>>>>>>
>>>>>> The issue we cannot ignore is that to check module-private
>>>>>> accessibility at compile-time means knowing the modules to which the
>>>>>> requesting and requested types belong. If I invoke:
>>>>>>
>>>>>> javac Foo.java Bar.java
>>>>>>
>>>>>> where Foo.java refers to Bar, then a compiler must be able to compute
>>>>>> at least the module name to which Foo belongs as Foo is compiled, if Bar is
>>>>>> found to be module-private as it is compiled. There are no deployment
>>>>>> artifacts around yet. It shouldn't be a surprise that Foo.class and
>>>>>> Bar.class are subsequently emitted to encode their module membership.
>>>>>>
>>>>>> So with module accessibility comes module membership. Module
>>>>>> dependencies are a quite different topic which we'll get to soon enough, and
>>>>>> they are the place where minimizing new concepts is essential.
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>> Peter Kriens wrote:
>>>>>>
>>>>>>> I think it would make sense to work from first principles
>>>>>>> 1. What problem are we trying to solve?
>>>>>>> 2. How do we scope this problem to an acceptable area for all?
>>>>>>> 3. What requirements should the solution fulfill
>>>>>>> I would like to have a short discussion about the problem so we are
>>>>>>> sure we are all on the same page. I can start off with the problem I think
>>>>>>> we are trying to solve.
>>>>>>> Modularity
>>>>>>> Modularity is the art of encapsulation and hiding. A module limits
>>>>>>> the amount of information outside its boundaries. This reduces the overall
>>>>>>> complexity of a system because it becomes possible to reason (and change)
>>>>>>> locally about a module instead of understanding the whole system, and it
>>>>>>> provides a local namespace that is easier to work with for humans than a
>>>>>>> global namespace.
>>>>>>> Java provides modularity in many places. A type encapsulates its
>>>>>>> fields and methods and selectively exposes them to other types with the
>>>>>>> access modifies: private, protected and public. A package encapsulates a
>>>>>>> number of types and resources. And last, but not least, the class loader
>>>>>>> provides a class space that can be distinct from other class spaces.
>>>>>>> The modularity that is enabled with class loaders has been exploited
>>>>>>> by many. However, the Java Language has little to say about class loaders;
>>>>>>> it is outside the scope of the language.
>>>>>>> So what problem do we need to solve, seeing that Java provides
>>>>>>> already so many forms of modularity? There is a sense in the community that
>>>>>>> packages are too small for modules. Though the package names hint at a
>>>>>>> hierarchy (and this is how they are found in a file system), this hierarchy
>>>>>>> does not provide a preferential treatment for children. Many programs
>>>>>>> consist of hundreds of packages. However, except for the limited package
>>>>>>> concept, the programmer cannot indicate that the visibility of an artifact
>>>>>>> should be limited to the program, or part of the program.
>>>>>>> A large number of delivery formats have been created over the years,
>>>>>>> most of the based on the JAR format (which is also not a part of the
>>>>>>> language): WAR, EAR, OSGi bundles, midlets, xlets, and likely many more
>>>>>>> proprietary formats. Usually, the modularity of these modules is based on a
>>>>>>> rather simplistic hierarchical class loader model. That is, all classes in
>>>>>>> ancestors are visible, but not any siblings or their children. The language
>>>>>>> is moot on the point of this type of encapsulation.
>>>>>>> Delivery modules contain types that depend on types in other delivery
>>>>>>> modules. There is no uniformly agreed Java standard to model these
>>>>>>> dependencies. During build time, the compiler is provided with a linear list
>>>>>>> of JARs and the compiler picks the first type that matches a name. Only the
>>>>>>> name is encoded in the class file, not the originating delivery module.
>>>>>>> During runtime, the same process is used to find classes, albeit in a
>>>>>>> hierarchical class loader model. This has all so far been outside the Java
>>>>>>> Language.
>>>>>>> It is crucial that we distinguish the language/logic modularity and
>>>>>>> the modularity based on the deployment artifact. Interestingly, in my
>>>>>>> understanding, .NET makes such a distinction between the modularity of a
>>>>>>> delivery unit (called an assembly) and the finer grained modularity inside a
>>>>>>> delivery unit, represented by namespaces. The "internal" keyword indicates
>>>>>>> that the artifact is visible only inside an assembly and the "namespace"
>>>>>>> keyword provides a hierarchical namespace.
>>>>>>> Therefore, one can identify two problems in the Java platform
>>>>>>> concerning modularity
>>>>>>> 1. Packages are too limited for proper language/logic modularity
>>>>>>> 2. Lack of a runtime module concept that is related to deployment
>>>>>>>   artifacts (physical modularity)
>>>>>>> Looking at the proposed time frame (EDR by mid-April), attacking both
>>>>>>> problems simultaneously seems rather unrealistic based on my limited
>>>>>>> experience. I therefore suggest to start with problem #1: Packages are too
>>>>>>> limited for proper language/logic modularity. What do you think?
>>>>>>> Kind regards,
>>>>>>> Peter Kriens
>>>>>>> On 27 jan 2009, at 05:31, Alex Buckley wrote:
>>>>>>>
>>>>>>>> Comments welcome.
>>>>>>>>
>>>>>>>> 1) Packages are typically arranged in hierarchies, but types can
>>>>>>>> only be
>>>>>>>> used across different branches of the hierarchy by being made
>>>>>>>> public,
>>>>>>>> which exposes implementation details too widely. Information hiding
>>>>>>>> is
>>>>>>>> further reduced by interface members always being public.
>>>>>>>>
>>>>>>>> Non-solution: hierarchical package membership. Redefining existing
>>>>>>>> well-known semantics is always a bad idea, and this one is
>>>>>>>> especially
>>>>>>>> complicated. There would need to be a way to stop package-private
>>>>>>>> artifacts from being accessible to subpackages, or else we would be
>>>>>>>> strengthening information hiding in one place only to weaken it
>>>>>>>> elsewhere. Also, there would need to be a way to configure the depth
>>>>>>>> of
>>>>>>>> exposure of package-private artifacts, since artifacts in package A
>>>>>>>> should sometimes be accessible to A.B and not to A.B.C, and so on.
>>>>>>>>
>>>>>>>> 2) Dependencies of one type on another are expressed in source and
>>>>>>>> classfiles, but there is no standard way for a Java compiler or JVM
>>>>>>>> to
>>>>>>>> interact with its environment to read and resolve dependencies. This
>>>>>>>> causes compile-time and runtime environments to differ, and
>>>>>>>> complicates
>>>>>>>> any effort to version types available in the (compile-time or
>>>>>>>> runtime)
>>>>>>>> environment.
>>>>>>>>
>>>>>>>> Non-solution: standardize the CLASSPATH. One list for all programs
>>>>>>>> and
>>>>>>>> libraries in the JVM makes versioning almost meaningless, and
>>>>>>>> packages
>>>>>>>> whose types occur in multiple CLASSPATH entries can behave poorly.
>>>>>>>> (See
>>>>>>>> Peter Kriens' presentation at Devoxx 2008.)
>>>>>>>>
>>>>>>>> 3) Many packages have grown large over the years. They are
>>>>>>>> effectively
>>>>>>>> impossible to refactor now, since it is binary-incompatible to
>>>>>>>> rename
>>>>>>>> types and dangerous to "split" packages. The next-best option is to
>>>>>>>> let
>>>>>>>> subsets be taken of existing packages, and deliver subsets
>>>>>>>> independently, but there is no mechanism in the Java language or JVM
>>>>>>>> for
>>>>>>>> that.
>>>>>>>>
>>>>>>>> Non-solution: a mechanism for renaming packages and types outside
>>>>>>>> the
>>>>>>>> Java language. This would require updating Java's model of binary
>>>>>>>> compatibility, and would make source code incomprehensible.
>>>>>>>>
>>>>>>>> Alex
>>>>>>>> _______________________________________________
>>>>>>>> jsr294-modularity-eg mailing list
>>>>>>>> jsr294-modularity-eg at cs.oswego.edu <mailto:
>>>>>>>> jsr294-modularity-eg at cs.oswego.edu>
>>>>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>> _______________________________________________
>>>>>>> jsr294-modularity-eg mailing list
>>>>>>> jsr294-modularity-eg at cs.oswego.edu
>>>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>>>>
>>>>>> _______________________________________________
>>>>>> jsr294-modularity-eg mailing list
>>>>>> jsr294-modularity-eg at cs.oswego.edu
>>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>>>
>>>>> _______________________________________________
>>>>> jsr294-modularity-eg mailing list
>>>>> jsr294-modularity-eg at cs.oswego.edu
>>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>>
>>>> _______________________________________________
>>>> jsr294-modularity-eg mailing list
>>>> jsr294-modularity-eg at cs.oswego.edu
>>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>>
>>> _______________________________________________
>>> jsr294-modularity-eg mailing list
>>> jsr294-modularity-eg at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/jsr294-modularity-observer/attachments/20090213/b61a6ecd/attachment-0001.html>

More information about the jsr294-modularity-observer mailing list