[jsr294-modularity-eg] Module boundaries
Richard S. Hall
heavy at ungoverned.org
Fri Feb 13 19:59:29 EST 2009
Chris,
I am sure everyone wants a "simple" solution, but one person's "simple"
is another's "simplistic" and yet another's "complicated". That's the
debate we need to have. :-)
Regarding the masquerading issue, I think it falls more into the
security realm. It is hard to get any guarantees without security
enabled (and even then it is difficult).
-> richard
Christopher Brind wrote:
> Hi Richard,
>
> So you are saying that masquerading is acceptable and JSR 294 is not
> going to even try to prevent it?
>
> If that is the case, why is what is being proposed/discussed for JSR
> 294 so complicated?
>
> For instance, a simpler solution would be to create a new type
> modifier which bridges the gap between public and package private,
> allowing types with the modifier to be accessed by classes in the
> parent package only.
>
> e.g. for want of a better name lets call it 'internal' (I think I read
> internal suggested before, but can't think of a better name, sorry)
>
> uk/org/brindy/app/geo
> Geo: internal class Geo { ... new GeoSupport() ... }
> GeoSupport: class GeoSuport { ... }
>
> uk/org/brindy/app:
> Main: public class Main { ... new Geo() ... }
>
>
> That results in minimal changes to Java, is easy to understand and
> seems to 'fix' the problem described in the original JSR 294.
>
> Regards,
> Chris
>
>
> 2009/2/13 Richard S. Hall <heavy at ungoverned.org
> <mailto:heavy at ungoverned.org>>
>
> Isn't such masquerading possible in approaches that allow separate
> compilation? I am not sure this is an issue. For example, nothing
> stops me from creating a class in someone else's package and
> adding it to a JAR file to get package-private access.
>
> -> richard
>
> Christopher Brind wrote:
>
> Hi,
>
> I have to admit that I read the straw-man with a mixture of
> both excitement and trepidation. :)
>
> The first question that springs to mind is, what is to stop me
> from contributing to the 'Felix' module example in a way that
> then exposes some of the module private functionality, e.g.
> via a delegate?
>
> So, for example, let's say that I have downloaded a JAR which
> contains the Felix classes from the example in the straw-man.
> I could compile the following in order to contribute to the
> Felix module (source tree example like in the straw-man):
> module-info: module Felix;
>
> org/apache/felix/framework
> PublicFelix: package org.apache.felix.framework;
> public class PublicFelix { ... }
>
> I then repackage the downloaded JAR adding the PublicFelix
> class. The result is that because I told the compiler that
> the PublicFelix class is in the Felix module, it can access
> the Felix class. The code in the PublicFelix class is
> basically a delegate for the Felix class, but since the
> PublicFelix class is accessible from anywhere the ultimate
> result is that the module system has been circumvented.
>
> The implication of this seems to be that if you want to
> prevent this from happening then you will have to provide the
> binaries in a format which prevents tampering. For instance,
> if the original Felix module was compiled and then bundled as
> a signed JAR I could not contribute to the module as described
> above as that will break the JAR's signature. However, the
> proposal says that types in a module can be loaded from
> multiple containers, e.g. URL Class Loader. How do I protect
> my module's internal implementation in this case?
>
> Regards,
> Chris
>
>
>
> 2009/2/13 Peter Kriens <peter.kriens at aqute.biz
> <mailto:peter.kriens at aqute.biz> <mailto:peter.kriens at aqute.biz
> <mailto:peter.kriens at aqute.biz>>>
>
>
> Enclosed find the first strawman proposal for nested
> modules. This
> was not easy :-( but I tried hard to make it as clear as
> possible.
> I guess sending docs will probably not work or create too much
> CO2, so here is the link:
> http://www.aqute.biz/JSR294/HomePage. Enjoy.
>
> Kind regards,
>
> Peter Kriens
>
>
> On 4 feb 2009, at 10:32, Alex Buckley wrote:
>
> Yes, setting module membership at runtime effectively
> simulates 'internal', where the .NET assembly-build
> operation
> is done lazily.
>
> Module nesting sounds interesting. Could you please
> send some
> source code which exhibits how it would impact
> module-private
> accessibility?
>
> We can certainly have a phone call. I am travelling
> internationally for the next week and, in practice,
> will not
> be back in the office until Tuesday 17th.
>
> EG members, please indicate when on 17-20th February
> you could
> be available. Good times would be 9am West Coast, 12pm East
> Coast, 6pm France.
>
> Alex
>
> Peter Kriens wrote:
>
> Ok, so you allow a module boundary to be set in
> runtime,
> this will make the module keyword in unbound modules
> reflect the semantics of the .NET 'internal' concept? I
> agree we should then have a discussion later about
> how the
> compiler gets the unbound module boundaries.(Do we
> have an
> action list for these things?)
> If I look at software projects than there will be a
> tendency to put a set of related packages into a bound
> module, where a number of these bound modules are
> packaged
> in a JAR (the unbound module). This seems to scream for
> nesting of bound modules?
> VM -> unbound module -> bound module +-> package
> -> type -+
> ^ | ^
> |
> +--------------+
> +--------+
> And, can we please, pretty please, have a kick-off
> phone
> conference now we're a group starting to work on this?
> Kind regards,
> Peter Kriens
> On 3 feb 2009, at 01:47, Alex Buckley wrote:
>
> .NET is not Java.
>
> Compiled types in .NET live in non-executable
> modules
> (.netmodule files). To be executed, modules must be
> packaged into assemblies (.dll/.exe files).
> This packaging makes "internal" a bit tricky.
> "internal" is specified as "accessible from the
> same
> program", where a program is merely a bunch of
> source
> files. A C# compiler must take it on trust that the
> .netmodule which accesses an "internal"
> artifact will
> be packaged in the same assembly as the .netmodule
> which declares the artifact.
>
> This is not unreasonable, since .netmodules MUST be
> packaged. But if packaged "wrongly", code
> accessing an
> "internal" artifact which the compiler promised was
> accessible will fail at runtime. You do not want
> language features to behave like this. There is
> effectively a hole in the C# spec caused by the
> greater CLI spec stratifying loadable code into
> non-executable and executable.
>
> Java is different. A JVM does not require
> classfiles
> to be packaged in any way before executing
> them. Nor
> does the reference implementation of a Java
> compiler
> require classfiles to be packaged before compiling
> against them. These things are not going to change.
> Since module-private accessibility is a
> language and
> VM construct, it cannot depend on the future
> packaging
> of code still being compiled or compiled against.
>
> (What it can depend on is an API consulted by a
> Java
> compiler to discover whether module M version X and
> module M version Y have "equal" versions. The API
> implementation would be honor-bound to give
> consistent
> answers, similar to classloaders. We'll get to that
> API soon, I hope.)
>
> This boils down to saying the module-private
> accessibility and module membership go hand-in-hand
> for me. In any case, the first thing a
> programmer will
> ask when seeing a module-private member is: "What
> module does this member belong to?". The answer
> should
> NOT be "it depends on a compiler flag or
> invocation of
> jar".
>
> I will shortly send a Java language grammar and
> classfile format which reflect these views.
>
> Now, I know this issue is not going to go away. I
> therefore propose that IN ADDITION to encoding
> module
> membership in source, it should be possible for the
> host system to determine module membership. If you
> REALLY want the answer above to be "it depends on a
> compiler flag or invocation of jar", you can do
> that.
> But a Java compiler/VM must defer to the
> membership in
> a source/class file if it exists.
>
> Is that enough of a weakening of my requirements #1
> and #2?
>
> Alex
>
> Peter Kriens wrote:
>
> Hmm. .NET shows that you do not have to
> know all
> the module boundaries during compilation.
> 'internal' indicates inside the assembly
> and the
> source assembly is clearly not known during
> compilation. This concept looks attractively
> simple and is in line with common practice.
> For 294, during compilation, the JAR files and
> project source can easily be used inside the
> compiler for the scope. During runtime, the
> module
> system can set this boundary.
> However, I also like a module concept that is
> defined in the language. This will be more
> complex
> (and probably should be hierarchical) but it
> should be not attempt to define the deployment
> artifact nor dependencies. That is, it
> should be
> more like namespaces.
> Kind regards,
> Peter Kriens
> On 28 jan 2009, at 03:56, Alex Buckley wrote:
>
> I am happy to discuss module-private
> accessibility first, in that it
> addresses the
> limitations of packages on which we all
> agree.
>
> The issue we cannot ignore is that to check
> module-private accessibility at
> compile-time
> means knowing the modules to which the
> requesting and requested types belong. If I
> invoke:
>
> javac Foo.java Bar.java
>
> where Foo.java refers to Bar, then a
> compiler
> must be able to compute at least the module
> name to which Foo belongs as Foo is
> compiled,
> if Bar is found to be module-private as
> it is
> compiled. There are no deployment artifacts
> around yet. It shouldn't be a surprise that
> Foo.class and Bar.class are subsequently
> emitted to encode their module membership.
>
> So with module accessibility comes module
> membership. Module dependencies are a quite
> different topic which we'll get to soon
> enough, and they are the place where
> minimizing new concepts is essential.
>
> Alex
>
> Peter Kriens wrote:
>
> I think it would make sense to work
> from
> first principles
> 1. What problem are we trying to solve?
> 2. How do we scope this problem to an
> acceptable area for all?
> 3. What requirements should the
> solution
> fulfill
> I would like to have a short discussion
> about the problem so we are sure we are
> all on the same page. I can start
> off with
> the problem I think we are trying
> to solve.
> Modularity
> Modularity is the art of
> encapsulation and
> hiding. A module limits the amount of
> information outside its boundaries.
> This
> reduces the overall complexity of a
> system
> because it becomes possible to
> reason (and
> change) locally about a module
> instead of
> understanding the whole system, and it
> provides a local namespace that is
> easier
> to work with for humans than a global
> namespace.
> Java provides modularity in many
> places. A
> type encapsulates its fields and
> methods
> and selectively exposes them to other
> types with the access modifies:
> private,
> protected and public. A package
> encapsulates a number of types and
> resources. And last, but not least, the
> class loader provides a class space
> that
> can be distinct from other class
> spaces.
> The modularity that is enabled with
> class
> loaders has been exploited by many.
> However, the Java Language has
> little to
> say about class loaders; it is
> outside the
> scope of the language.
> So what problem do we need to solve,
> seeing that Java provides already
> so many
> forms of modularity? There is a
> sense in
> the community that packages are too
> small
> for modules. Though the package
> names hint
> at a hierarchy (and this is how
> they are
> found in a file system), this hierarchy
> does not provide a preferential
> treatment
> for children. Many programs consist of
> hundreds of packages. However,
> except for
> the limited package concept, the
> programmer cannot indicate that the
> visibility of an artifact should be
> limited to the program, or part of the
> program.
> A large number of delivery formats have
> been created over the years, most
> of the
> based on the JAR format (which is
> also not
> a part of the language): WAR, EAR, OSGi
> bundles, midlets, xlets, and likely
> many
> more proprietary formats. Usually, the
> modularity of these modules is
> based on a
> rather simplistic hierarchical class
> loader model. That is, all classes in
> ancestors are visible, but not any
> siblings or their children. The
> language
> is moot on the point of this type of
> encapsulation.
> Delivery modules contain types that
> depend
> on types in other delivery modules.
> There
> is no uniformly agreed Java standard to
> model these dependencies. During build
> time, the compiler is provided with a
> linear list of JARs and the
> compiler picks
> the first type that matches a name.
> Only
> the name is encoded in the class
> file, not
> the originating delivery module. During
> runtime, the same process is used
> to find
> classes, albeit in a hierarchical class
> loader model. This has all so far been
> outside the Java Language.
> It is crucial that we distinguish the
> language/logic modularity and the
> modularity based on the deployment
> artifact. Interestingly, in my
> understanding, .NET makes such a
> distinction between the modularity of a
> delivery unit (called an assembly)
> and the
> finer grained modularity inside a
> delivery
> unit, represented by namespaces. The
> "internal" keyword indicates that the
> artifact is visible only inside an
> assembly and the "namespace" keyword
> provides a hierarchical namespace.
> Therefore, one can identify two
> problems
> in the Java platform concerning
> modularity
> 1. Packages are too limited for proper
> language/logic modularity
> 2. Lack of a runtime module concept
> that
> is related to deployment
> artifacts (physical modularity)
> Looking at the proposed time frame
> (EDR by
> mid-April), attacking both problems
> simultaneously seems rather unrealistic
> based on my limited experience. I
> therefore suggest to start with problem
> #1: Packages are too limited for proper
> language/logic modularity. What do
> you think?
> Kind regards,
> Peter Kriens
> On 27 jan 2009, at 05:31, Alex
> Buckley wrote:
>
> Comments welcome.
>
> 1) Packages are typically
> arranged in
> hierarchies, but types can only be
> used across different branches
> of the
> hierarchy by being made public,
> which exposes implementation
> details
> too widely. Information hiding is
> further reduced by interface
> members
> always being public.
>
> Non-solution: hierarchical package
> membership. Redefining existing
> well-known semantics is always
> a bad
> idea, and this one is especially
> complicated. There would need
> to be a
> way to stop package-private
> artifacts from being accessible to
> subpackages, or else we would be
> strengthening information hiding in
> one place only to weaken it
> elsewhere. Also, there would
> need to
> be a way to configure the depth of
> exposure of package-private
> artifacts,
> since artifacts in package A
> should sometimes be accessible
> to A.B
> and not to A.B.C, and so on.
>
> 2) Dependencies of one type on
> another
> are expressed in source and
> classfiles, but there is no
> standard
> way for a Java compiler or JVM to
> interact with its environment
> to read
> and resolve dependencies. This
> causes compile-time and runtime
> environments to differ, and
> complicates
> any effort to version types
> available
> in the (compile-time or runtime)
> environment.
>
> Non-solution: standardize the
> CLASSPATH. One list for all
> programs and
> libraries in the JVM makes
> versioning
> almost meaningless, and packages
> whose types occur in multiple
> CLASSPATH entries can behave
> poorly. (See
> Peter Kriens' presentation at
> Devoxx
> 2008.)
>
> 3) Many packages have grown
> large over
> the years. They are effectively
> impossible to refactor now,
> since it
> is binary-incompatible to rename
> types and dangerous to "split"
> packages. The next-best option
> is to let
> subsets be taken of existing
> packages,
> and deliver subsets
> independently, but there is no
> mechanism in the Java language
> or JVM for
> that.
>
> Non-solution: a mechanism for
> renaming
> packages and types outside the
> Java language. This would require
> updating Java's model of binary
> compatibility, and would make
> source
> code incomprehensible.
>
> Alex
>
> _______________________________________________
> jsr294-modularity-eg mailing list
>
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>
> _______________________________________________
> jsr294-modularity-eg mailing list
> jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>
> <mailto:jsr294-modularity-eg at cs.oswego.edu
> <mailto:jsr294-modularity-eg at cs.oswego.edu>>
> http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-eg
>
>
>
More information about the jsr294-modularity-observer
mailing list