[jsr294-modularity-eg] Module boundaries
Christopher Brind
brindy at brindy.org.uk
Sat Feb 14 05:54:19 EST 2009
Hi Richard,
Actually, this is the debate we're having. =) Or do you mean the expert
group needs to have the debate?
In my view JSR 294 itself is broken. Either the title is wrong, "JSR 294:
Improved Modularity Support in the JavaTM Programming Language" or the
problem specification is wrong:
> Today, an implementation can be partitioned into multiple packages.
> Subparts of such an implementation need to be more tightly coupled to each
> other than to the surrounding software environment. Today designers are
> forced to declare elements of the program that are needed by other subparts
> of the implementation as public - thereby making them globally accessible,
> which is clearly suboptimal.
>
> Alternately, the entire implementation can be placed in a single package.
> This resolves the issue above, but is unwieldy, and exposes all internals of
> all subparts to each other.
>
> The language changes envisioned will resolve these issues. In particular,
> we expect to introduce a new notion of modules (superpackages) at the
> language level where existing public access control would apply only within
> a language level module and access to API's from outside a module would be
> restricted to API's the module explicitly exports.
>
Which as you can see doesn't explicitly mention modularity until the last
paragraph, which is in actual fact a solution to the problem stated in the
previous two paragraphs. It seems like a bit of leap to go from a missing
type modifier (which is what seems to be being described) to modularity.
If 'internal' were available tomorrow, I could and would use it. If modules
(as have been described to date) were available tomorrow, I know I wouldn't
use them and would continue to use Java 'as is' allowing specifications such
as OSGi to handle modularity for me, mainly because the added complexity of
modules doesn't seem to add a significant amount of benefit at a language
level for the level of complexity.
Humbly,
Chris
2009/2/14 Richard S. Hall <heavy at ungoverned.org>
> Chris,
>
> I am sure everyone wants a "simple" solution, but one person's "simple" is
> another's "simplistic" and yet another's "complicated". That's the debate we
> need to have. :-)
>
> Regarding the masquerading issue, I think it falls more into the security
> realm. It is hard to get any guarantees without security enabled (and even
> then it is difficult).
>
> -> richard
>
> Christopher Brind wrote:
>
>> Hi Richard,
>>
>> So you are saying that masquerading is acceptable and JSR 294 is not going
>> to even try to prevent it?
>>
>> If that is the case, why is what is being proposed/discussed for JSR 294
>> so complicated?
>> For instance, a simpler solution would be to create a new type modifier
>> which bridges the gap between public and package private, allowing types
>> with the modifier to be accessed by classes in the parent package only.
>>
>> e.g. for want of a better name lets call it 'internal' (I think I read
>> internal suggested before, but can't think of a better name, sorry)
>>
>> uk/org/brindy/app/geo
>> Geo: internal class Geo { ... new GeoSupport() ... }
>> GeoSupport: class GeoSuport { ... }
>>
>> uk/org/brindy/app:
>> Main: public class Main { ... new Geo() ... }
>>
>>
>> That results in minimal changes to Java, is easy to understand and seems
>> to 'fix' the problem described in the original JSR 294.
>>
>> Regards,
>> Chris
>>
>>
>> 2009/2/13 Richard S. Hall <heavy at ungoverned.org <mailto:
>> heavy at ungoverned.org>>
>>
>>
>> Isn't such masquerading possible in approaches that allow separate
>> compilation? I am not sure this is an issue. For example, nothing
>> stops me from creating a class in someone else's package and
>> adding it to a JAR file to get package-private access.
>>
>> -> richard
>>
>> Christopher Brind wrote:
>>
>> Hi,
>>
>> I have to admit that I read the straw-man with a mixture of
>> both excitement and trepidation. :)
>>
>> The first question that springs to mind is, what is to stop me
>> from contributing to the 'Felix' module example in a way that
>> then exposes some of the module private functionality, e.g.
>> via a delegate?
>>
>> So, for example, let's say that I have downloaded a JAR which
>> contains the Felix classes from the example in the straw-man.
>> I could compile the following in order to contribute to the
>> Felix module (source tree example like in the straw-man):
>> module-info: module Felix;
>>
>> org/apache/felix/framework
>> PublicFelix: package org.apache.felix.framework;
>> public class PublicFelix { ... }
>>
>> I then repackage the downloaded JAR adding the PublicFelix
>> class. The result is that because I told the compiler that
>> the PublicFelix class is in the Felix module, it can access
>> the Felix class. The code in the PublicFelix class is
>> basically a delegate for the Felix class, but since the
>> PublicFelix class is accessible from anywhere the ultimate
>> result is that the module system has been circumvented.
>>
>> The implication of this seems to be that if you want to
>> prevent this from happening then you will have to provide the
>> binaries in a format which prevents tampering. For instance,
>> if the original Felix module was compiled and then bundled as
>> a signed JAR I could not contribute to the module as described
>> above as that will break the JAR's signature. However, the
>> proposal says that types in a module can be loaded from
>> multiple containers, e.g. URL Class Loader. How do I protect
>> my module's internal implementation in this case?
>>
>> Regards,
>> Chris
>>
>>
>>
>> 2009/2/13 Peter Kriens <peter.kriens at aqute.biz
>> <mailto:peter.kriens at aqute.biz> <mailto:peter.kriens at aqute.biz
>>
>> <mailto:peter.kriens at aqute.biz>>>
>>
>>
>> Enclosed find the first strawman proposal for nested
>> modules. This
>> was not easy :-( but I tried hard to make it as clear as
>> possible.
>> I guess sending docs will probably not work or create too much
>> CO2, so here is the link:
>> http://www.aqute.biz/JSR294/HomePage. Enjoy.
>>
>> Kind regards,
>>
>> Peter Kriens
>>
>>
>> On 4 feb 2009, at 10:32, Alex Buckley wrote:
>>
>> Yes, setting module membership at runtime effectively
>> simulates 'internal', where the .NET assembly-build
>> operation
>> is done lazily.
>>
>> Module nesting sounds interesting. Could you please
>> send some
>> source code which exhibits how it would impact
>> module-private
>> accessibility?
>>
>> We can certainly have a phone call. I am travelling
>> internationally for the next week and, in practice,
>> will not
>> be back in the office until Tuesday 17th.
>>
>> EG members, please indicate when on 17-20th February
>> you could
>> be available. Good times would be 9am West Coast, 12pm East
>> Coast, 6pm France.
>>
>> Alex
>>
>> Peter Kriens wrote:
>>
>> Ok, so you allow a module boundary to be set in
>> runtime,
>> this will make the module keyword in unbound modules
>> reflect the semantics of the .NET 'internal' concept? I
>> agree we should then have a discussion later about
>> how the
>> compiler gets the unbound module boundaries.(Do we
>> have an
>> action list for these things?)
>> If I look at software projects than there will be a
>> tendency to put a set of related packages into a bound
>> module, where a number of these bound modules are
>> packaged
>> in a JAR (the unbound module). This seems to scream for
>> nesting of bound modules?
>> VM -> unbound module -> bound module +-> package
>> -> type -+
>> ^ | ^
>> |
>> +--------------+
>> +--------+
>> And, can we please, pretty please, have a kick-off
>> phone
>> conference now we're a group starting to work on this?
>> Kind regards,
>> Peter Kriens
>> On 3 feb 2009, at 01:47, Alex Buckley wrote:
>>
>> .NET is not Java.
>>
>> Compiled types in .NET live in non-executable
>> modules
>> (.netmodule files). To be executed, modules must be
>> packaged into assemblies (.dll/.exe files).
>> This packaging makes "internal" a bit tricky.
>> "internal" is specified as "accessible from the
>> same
>> program", where a program is merely a bunch of
>> source
>> files. A C# compiler must take it on trust that the
>> .netmodule which accesses an "internal"
>> artifact will
>> be packaged in the same assembly as the .netmodule
>> which declares the artifact.
>>
>> This is not unreasonable, since .netmodules MUST be
>> packaged. But if packaged "wrongly", code
>> accessing an
>> "internal" artifact which the compiler promised was
>> accessible will fail at runtime. You do not want
>> language features to behave like this. There is
>> effectively a hole in the C# spec caused by the
>> greater CLI spec stratifying loadable code into
>> non-executable and executable.
>>
>> Java is different. A JVM does not require
>> classfiles
>> to be packaged in any way before executing
>> them. Nor
>> does the reference implementation of a Java
>> compiler
>> require classfiles to be packaged before compiling
>> against them. These things are not going to change.
>> Since module-private accessibility is a
>> language and
>> VM construct, it cannot depend on the future
>> packaging
>> of code still being compiled or compiled against.
>>
>> (What it can depend on is an API consulted by a
>> Java
>> compiler to discover whether module M version X and
>> module M version Y have "equal" versions. The API
>> implementation would be honor-bound to give
>> consistent
>> answers, similar to classloaders. We'll get to that
>> API soon, I hope.)
>>
>> This boils down to saying the module-private
>> accessibility and module membership go hand-in-hand
>> for me. In any case, the first thing a
>> programmer will
>> ask when seeing a module-private member is: "What
>> module does this member belong to?". The answer
>> should
>> NOT be "it depends on a compiler flag or
>> invocation of
>> jar".
>>
>> I will shortly send a Java language grammar and
>> classfile format which reflect these views.
>>
>> Now, I know this issue is not going to go away. I
>> therefore propose that IN ADDITION to encoding
>> module
>> membership in source, it should be possible for the
>> host system to determine module membership. If you
>> REALLY want the answer above to be "it depends on a
>> compiler flag or invocation of jar", you can do
>> that.
>> But a Java compiler/VM must defer to the
>> membership in
>> a source/class file if it exists.
>>
>> Is that enough of a weakening of my requirements #1
>> and #2?
>>
>> Alex
>>
>> Peter Kriens wrote:
>>
>> Hmm. .NET shows that you do not have to
>> know all
>> the module boundaries during compilation.
>> 'internal' indicates inside the assembly
>> and the
>> source assembly is clearly not known during
>> compilation. This concept looks attractively
>> simple and is in line with common practice.
>> For 294, during compilation, the JAR files and
>> project source can easily be used inside the
>> compiler for the scope. During runtime, the
>> module
>> system can set this boundary.
>> However, I also like a module concept that is
>> defined in the language. This will be more
>> complex
>> (and probably should be hierarchical) but it
>> should be not attempt to define the deployment
>> artifact nor dependencies. That is, it
>> should be
>> more like namespaces.
>> Kind regards,
>> Peter Kriens
>> On 28 jan 2009, at 03:56, Alex Buckley wrote:
>>
>> I am happy to discuss module-private
>> accessibility first, in that it
>> addresses the
>> limitations of packages on which we all
>> agree.
>>
>> The issue we cannot ignore is that to check
>> module-private accessibility at
>> compile-time
>> means knowing the modules to which the
>> requesting and requested types belong. If I
>> invoke:
>>
>> javac Foo.java Bar.java
>>
>> where Foo.java refers to Bar, then a
>> compiler
>> must be able to compute at least the module
>> name to which Foo belongs as Foo is
>> compiled,
>> if Bar is found to be module-private as
>> it is
>> compiled. There are no deployment artifacts
>> around yet. It shouldn't be a surprise that
>> Foo.class and Bar.class are subsequently
>> emitted to encode their module membership.
>>
>> So with module accessibility comes module
>> membership. Module dependencies are a quite
>> different topic which we'll get to soon
>> enough, and they are the place where
>> minimizing new concepts is essential.
>>
>> Alex
>>
>> Peter Kriens wrote:
>>
>> I think it would make sense to work
>> from
>> first principles
>> 1. What problem are we trying to solve?
>> 2. How do we scope this problem to an
>> acceptable area for all?
>> 3. What requirements should the
>> solution
>> fulfill
>> I would like to have a short discussion
>> about the problem so we are sure we are
>> all on the same page. I can start
>> off with
>> the problem I think we are trying
>> to solve.
>> Modularity
>> Modularity is the art of
>> encapsulation and
>> hiding. A module limits the amount of
>> information outside its boundaries.
>> This
>> reduces the overall complexity of a
>> system
>> because it becomes possible to
>> reason (and
>> change) locally about a module
>> instead of
>> understanding the whole system, and it
>> provides a local namespace that is
>> easier
>> to work with for humans than a global
>> namespace.
>> Java provides modularity in many
>> places. A
>> type encapsulates its fields and
>> methods
>> and selectively exposes them to other
>> types with the access modifies:
>> private,
>> protected and public. A package
>> encapsulates a number of types and
>> resources. And last, but not least, the
>> class loader provides a class space
>> that
>> can be distinct from other class
>> spaces.
>> The modularity that is enabled with
>> class
>> loaders has been exploited by many.
>> However, the Java Language has
>> little to
>> say about class loaders; it is
>> outside the
>> scope of the language.
>> So what problem do we need to solve,
>> seeing that Java provides already
>> so many
>> forms of modularity? There is a
>> sense in
>> the community that packages are too
>> small
>> for modules. Though the package
>> names hint
>> at a hierarchy (and this is how
>> they are
>> found in a file system), this hierarchy
>> does not provide a preferential
>> treatment
>> for children. Many programs consist of
>> hundreds of packages. However,
>> except for
>> the limited package concept, the
>> programmer cannot indicate that the
>> visibility of an artifact should be
>> limited to the program, or part of the
>> program.
>> A large number of delivery formats have
>> been created over the years, most
>> of the
>> based on the JAR format (which is
>> also not
>> a part of the language): WAR, EAR, OSGi
>> bundles, midlets, xlets, and likely
>> many
>> more proprietary formats. Usually, the
>> modularity of these modules is
>> based on a
>> rather simplistic hierarchical class
>> loader model. That is, all classes in
>> ancestors are visible, but not any
>> siblings or their children. The
>> language
>> is moot on the point of this type of
>> encapsulation.
>> Delivery modules contain types that
>> depend
>> on types in other delivery modules.
>> There
>> is no uniformly agreed Java standard to
>> model these dependencies. During build
>> time, the compiler is provided with a
>> linear list of JARs and the
>> compiler picks
>> the first type that matches a name.
>> Only
>> the name is encoded in the class
>> file, not
>> the originating delivery module. During
>> runtime, the same process is used
>> to find
>> classes, albeit in a hierarchical class
>> loader model. This has all so far been
>> outside the Java Language.
>> It is crucial that we distinguish the
>> language/logic modularity and the
>> modularity based on the deployment
>> artifact. Interestingly, in my
>> understanding, .NET makes such a
>> distinction between the modularity of a
>> delivery unit (called an assembly)
>> and the
>> finer grained modularity inside a
>> delivery
>> unit, represented by namespaces. The
>> "internal" keyword indicates that the
>> artifact is visible only inside an
>> assembly and the "namespace" keyword
>> provides a hierarchical namespace.
>> Therefore, one can identify two
>> problems
>> in the Java platform concerning
>> modularity
>> 1. Packages are too limited for proper
>> language/logic modularity
>> 2. Lack of a runtime module concept
>> that
>> is related to deployment
>> artifacts (physical modularity)
>> Looking at the proposed time frame
>> (EDR by
>> mid-April), attacking both problems
>> simultaneously seems rather unrealistic
>> based on my limited experience. I
>> therefore suggest to start with problem
>> #1: Packages are too limited for proper
>> language/logic modularity. What do
>> you think?
>> Kind regards,
>> Peter Kriens
>> On 27 jan 2009, at 05:31, Alex
>> Buckley wrote:
>>
>> Comments welcome.
>>
>> 1) Packages are typically
>> arranged in
>> hierarchies, but types can only be
>> used across different branches
>> of the
>> hierarchy by being made public,
>> which exposes implementation
>> details
>> too widely. Information hiding is
>> further reduced by interface
>> members
>> always being public.
>>
>> Non-solution: hierarchical package
>> membership. Redefining existing
>> well-known semantics is always
>> a bad
>> idea, and this one is especially
>> complicated. There would need
>> to be a
>> way to stop package-private
>> artifacts from being accessible to
>> subpackages, or else we would be
>> strengthening information hiding in
>> one place only to weaken it
>> elsewhere. Also, there would
>> need to
>> be a way to configure the depth of
>> exposure of package-private
>> artifacts,
>> since artifacts in package A
>> should sometimes be accessible
>> to A.B
>> and not to A.B.C, and so on.
>>
>> 2) Dependencies of one type on
>> another
>> are expressed in source and
>> classfiles, but there is no
>> standard
>> way for a Java compiler or JVM to
>> interact with its environment
>> to read
>> and resolve dependencies. This
>> causes compile-time and runtime
>> environments to differ, and
>> complicates
>> any effort to version types
>> available
>> in the (compile-time or runtime)
>> environment.
>>
>> Non-solution: standardize the
>> CLASSPATH. One list for all
>> programs and
>> libraries in the JVM makes
>> versioning
>> almost meaningless, and packages
>> whose types occur in multiple
>> CLASSPATH entries can behave
>> poorly. (See
>> Peter Kriens' presentation at
>> Devoxx
>> 2008.)
>>
>> 3) Many packages have grown
>> large over
>> the years. They are effectively
>> impossible to refactor now,
>> since it
>> is binary-incompatible to rename
>> types and dangerous to "split"
>> packages. The next-best option
>> is to let
>> subsets be taken of existing
>> packages,
>> and deliver subsets
>> independently, but there is no
>> mechanism in the Java language
>> or JVM for
>> that.
>>
>> Non-solution: a mechanism for
>> renaming
>> packages and types outside the
>> Java language. This would require
>> updating Java's model of binary
>> compatibility, and would make
>> source
>> code incomprehensible.
>>
>> Alex
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>>
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>> <mailto:
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>>
>> _______________________________________________
>> jsr294-modularity-eg mailing list
>> jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>> <mailto:jsr294-modularity-eg at cs.oswego.edu
>> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/jsr294-modularity-observer/attachments/20090214/55fe02a4/attachment-0001.html>
More information about the jsr294-modularity-observer
mailing list