Hi Jim,
One set of questions inline below.
BTW, I think you're both right.
Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093
e-mail:
Jim Marino <> wrote on 06/18/2009 09:03:33 AM:
> [image removed]
>
> Re: [sca-assembly] [NEW ISSUE] Assembly specification unclear on
> Contribution vs Deployment - when can errors in artifacts be reported?
>
> Jim Marino
>
> to:
>
> Simon Nash
>
> 06/18/2009 09:04 AM
>
> Cc:
>
> OASIS Assembly
>
> hi,
>
> Sorry for the delay in responding ot this but I have been traveling
> and without a properly functioning internet connections. I do have
> comments inline but I have the feeling that we are lacking definitions
> of the various contribution states. I'll define two states to clarify
> my comments below:
>
> 1. When a contribution is in the installed state, its exports are
> visible in the domain and deployables are ready for deployment
I read your responses to Simon and they didn't answer the following questions.
What do you mean by ready for deployment?
Can you describe a composite that is not ready for deployment?
What about the case of adding deployment composites to installed contributions ala section 11.4.2 (CD03)? How does this play into your definition?
> 2. When a contribution is in the deployed state, its components are
> available to service a request on one or more runtimes in the domain
>
> Jim
>
> On Jun 5, 2009, at 4:51 AM, Simon Nash wrote:
>
> > See comments inline below.
> >
> > Simon
> >
> > Jim Marino wrote:
> >> On Jun 2, 2009, at 5:28 AM, Simon Nash wrote:
> >>> Jim Marino wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> > >> in sca-contribution.xml files within a Contribution. If an
> >>>>>> artifact > >> is deployed which has dependencies
> >>>>>> > >> on other artifacts, then those dependent artifacts are
> >>>>>> also deployed.
> >>>>>> > >> When the SCA runtime has one or more deployable artifacts,
> >>>>>> the > >> runtime attempts to put those artifacts
> >>>>>> > >> and any artifacts they depend on into the Running state.
> >>>>>> This can > >> fail due to errors in one or more of the
> >>>>>> artifacts
> >>>>>> > >> or the process can be delayed until all dependencies are
> >>>>>> available.
> >>>>>> > >> 11.3.1
> >>>>>> > >> Checking for errors in artifacts MUST NOT be done for
> >>>>>> artifacts in > >> the Installed state (ie where the artifacts
> >>>>>> are
> >>>>>> > >> simply part of installed contributions] {ASM120xx]
> >>>>>> > > This seems over-restrictive. For example, what about
> >>>>>> malformed > > artifacts
> >>>>>> > > such as .composite files with XML syntax errors? It might
> >>>>>> be useful > > to
> >>>>>> > > inform the user about such problems when the artifacts
> >>>>>> are > > installed, but
> >>>>>> > > the proposed rule would apparently prohibit an SCA runtime
> >>>>>> from even
> >>>>>> > > discovering these problems at installation time. I think a
> >>>>>> better > > rule
> >>>>>> > > would be to allow such errors to be detected as long as
> >>>>>> this does not
> >>>>>> > > prevent any artifacts from being deployed. For example:
> >>>>>> Any errors in
> >>>>>> > > artifacts in the Installed state (i.e., where the artifacts
> >>>>>> are part > > of
> >>>>>> > > installed contributions and have not been deployed) MUST
> >>>>>> NOT prevent > > the
> >>>>>> > > SCA runtime from deploying artifacts.
> >>>>>> > >
> >>>>>> >
> >>>>>> > I agree with the general statement that runtimes should be
> >>>>>> allowed to > detect syntactic errors upfront since that is
> >>>>>> what users would expect > and want (i.e. contributions should
> >>>>>> not be installed if they contain > invalid artifacts).
> >>>>>> However, I don't follow the part about not > preventing
> >>>>>> artifacts from being deployed. Is it the case that an error >
> >>>>>> in some random contribution artifact A, does not prevent a
> >>>>>> composite C > in the same contribution from being deployed?
> >>>>>> Many users would want to > prevent that scenario. For
> >>>>>> example, I would expect most users would > only want "clean"
> >>>>>> contributions in a production environment.
> >>>>>> >
> >>>>>> I think (I hope) what Simon means is that it's ok for a runtime
> >>>>>> to
> >>>>>> detect errors at install time (if possible and reasonable) as
> >>>>>> long as
> >>>>>> it's also possible to go ahead and deploy those artifacts even
> >>>>>> when
> >>>>>> the deployment engine knows there are errors in the artifacts.
> >>>>> In Jim's scenario above, if A has a known error but C does not
> >>>>> have
> >>>>> a known error, I think compliant runtimes should be allowed to
> >>>>> deploy
> >>>>> C if requested. I am leaning towards going further and saying
> >>>>> that
> >>>>> compliant runtimes should be required to deploy C if requested.
> >>>>>
> >>>> I think requiring runtimes to deploy artifacts from contributions
> >>>> that contain known errors would be very a bad thing to do. Here's
> >>>> why:
> >>>> 1. Most users have stringent regulation over production
> >>>> environments and do not permit application artifacts containing
> >>>> detectable errors to be deployed. In most cases I have seen, this
> >>>> also goes for development environments. For example, good
> >>>> development environments have mechanisms in place to verify
> >>>> artifacts such as integration tests that fail a build if an error
> >>>> is detected. This avoids accumulating broken artifacts and cruft
> >>>> in source repositories. Requiring a compliant runtime to make a
> >>>> contribution containing detectable errors available in a domain
> >>>> would negate being able to lock down a production or development
> >>>> environment in this way. 2. The random artifact A could be
> >>>> exported by the contribution. It is possible for a composite
> >>>> contained in another contribution to be deployed at some later
> >>>> date that references artifact A. The error in A will surface at
> >>>> that later point. I think most people would prefer the error to
> >>>> be raised at the time the contribution containing A was
> >>>> "installed" in the domain. 3. It's very difficult to calculate
> >>>> with absolute certainty that contribution C does not indirectly
> >>>> reference A, particularly if A can be located dynamically in
> >>>> code. Providing fail-fast behavior avoids runtime errors from
> >>>> surfacing where they can cause more damage. I do agree compliant
> >>>> runtimes should be allowed to deploy contributions containing
> >>>> invalid artifacts. However, they should not be required to do so.
> >>>>> A more interesting question is if another composite B has a known
> >>>>> error, should compliant runtimes be required or allowed to
> >>>>> deploy B
> >>>>> if requested? I think this depends on the kind of error. Some
> >>>>> errors might be severe enough to prevent the deployment from
> >>>>> succeeding. Some errors might be less severe so that it would be
> >>>>> possible to allow deployment, even though some of the resulting
> >>>>> components would not be executable. The SCA specs should define
> >>>>> which errors fall into which of these categories.
> >>>>>
> >>>> How can a runtime be certain an error in B will not cause a
> >>>> serious problem if B is deployed? Can you provide examples of
> >>>> such errors? Jim
> >>> An example would be a misplaced Java annotation. In this case, the
> >>> JavaCAA spec says that the SCA runtime MUST NOT run the component
> >>> which uses the invalid implementation code. This implies to me that
> >>> other components in the same contribution can be deployed and run.
> >>>
> >>> Simon
> >>>
> >>>
> >> It may be useful to break this down into two parts. First, are you
> >> saying that it should be possible for a runtime to selectively
> >> deploy a composite where some components are in error, i.e. only
> >> deploy the components contained in a composite that do not contain
> >> errors? I can't imagine a case, even in development, where this
> >> would be useful. I certainly can't imagine why someone would want
> >> this behavior in a production environment.
> > In this case, all the components would be deployed, but some of them
> > could not be run. So if component B has a wire to component C, both
> > B and C are deployed and the wire is valid. If B then makes an
> > invocation on C, this causes a runtime error. If B never makes an
> > invocation on C, B is allowed to run with errors.
> >
> I'm not sure I would define C as "deployed" in this scenario. I would
> describe deployment as the process of making components in a composite
> available to service requests on one or more runtimes. However, I
> don't think we need to argue that point at the moment. Regardless of
> how one defines deployment, in this scenario, B can't be deployed
> (using either definition of the term) if B's reference to C is
> required. Since a functioning wire cannot be established, it is not
> safe for the runtime to allow B to service requests. As a developer
> of a service, I would want to know upfront if a wire is not valid.
> Otherwise, there is no way to determine this without making a service
> invocation.
>
> > I see the above behaviour as one that runtimes should be allowed but
> > not required to support. IMO it would also be valid for a runtime
> > to fail the deployment of the composite containing B and C.
> >
> > Here's my proposal for what should be allowed and required.
> >
> > 1. Runtimes MAY allow contributions to be installed even though
> > they contain known errors. In this case, the runtime MAY detect
> > and report the errors, and MUST perform the installation.
> >
> I don't think the runtime should be required to proceed with the
> installation as it leads to the same problems I outlined previously.
> This seems to be in contradiction to statement 2 below, but I am
> probably reading it incorrectly. I think it is very important that a
> runtime be allowed to fail an installation if the contribution
> contains known errors. By contribution installation, I mean the
> process by which exported contribution artifacts become available for
> import by other contributions.
>
> > 2. Runtimes MAY prevent installation of contributions containing
> > known errors.
> >
> > 3. Some types of errors (defined by the spec) MUST be detected at
> > deployment time. In this case the composite containing the
> > error MUST NOT be deployed, and the installed contribution
> > containing the composite MUST NOT be uninstalled.
> >
> Sorry, I'm not sure I understand this. Is this saying that the runtime
> should not attempt uninstallation without operator intervention?
> Consider the following sequence:
>
> ---> install contribution ---> deploy composite ---> X error
>
> There needs to be a way to recover and for a contribution containing
> an erroneous composite to be uninstalled:
>
> ---> uninstall ---> install fixed contribution ---> deploy composite
> ---> success
>
> The uninstall operation should be a valid operation that can be
> performed by an administrator/deployer person.
>
> > 4. Some types of errors (defined by the spec) MAY be detected at
> > deployment time and otherwise MUST be detected at runtime.
> > If the error is detected at deployment time, the composite
> > containing the error MUST NOT be deployed, and the installed
> > contribution containing the composite MUST NOT be uninstalled.
> > If the error is detected at runtime, the deployed composite
> > containing the error MUST NOT be undeployed.
> >
>
> I think this is too vague. Some runtimes (like Fabric3) have the
> notion of transactional deployment. It should be permissible for
> "runtime" errors encountered during a deployment to result in a
> rollback of the domain state to what it was prior to the deployment
> operation.
>
> > 5. Some types of errors (defined by the spec) cannot be detected
> > at deployment time and MUST be detected at runtime. In this
> > case the deployed composite containing the error MUST NOT be
> > undeployed.
> >
> >> For the second part, we can assume the error is in an artifact not
> >> used by the composite being deployed. I'll start with a
> >> definition: when a contribution is in the installed state, its
> >> exports are visible in the domain and deployables are ready for
> >> deployment.
> >> I would expect the type of error identified above to be reported
> >> upfront when an attempt is made to install the contribution. How
> >> can a runtime be sure it is safe to install the contribution
> >> otherwise - i.e. latent errors are ruled out? The only way it is
> >> safe is if the Java class can never be used as a component
> >> implementation. This is very difficult to determine at best. In
> >> order for this to work, the runtime would need to use an algorithm
> >> similar to the following:
> >> 1. Determine if the Java class is directly referenceable from
> >> outside the contribution via an export/import. All applicable
> >> export/import types must be examined
> >> 2. Introspect all composites and their hierarchies (includes and
> >> implementation uses) in the contribution to calculate where the
> >> Java artifact is reachable from
> >> 3. Determine if any of the composites are referenceable from
> >> outside the contribution via an export/import. All applicable
> >> export/import types must be examined
> >> If any of these steps result in true, the runtime cannot guarantee
> >> safety. This algorithm becomes much more complex for other
> >> artifacts - for example, an artifact referenced by qname several
> >> layers deep in a hierarchy. It's also worth noting performing this
> >> calculation is likely to severely impact install time.
> >> Of course, a runtime could choose not to perform this check and
> >> install the contribution, throwing an error later if an attempt is
> >> made to use the artifact.
> > >
> > This would be my preference. I agree that trying to guarantee safety
> > wouldn't be practical.
> >
> > > However, I personally wouldn't want to use a runtime
> > that
> >> does that. Consider the following scenario:
> >> 1. Contribution A is installed with composite C and Java
> >> implementation J. A exports J's package. C does not reference J. J
> >> contains an error as described above.
> >> 2. C is deployed
> >> 3. At a later point in time, contribution B is installed containing
> >> composite C2. B imports J's package and the runtime resolves the
> >> import to contribution A's export. C2 uses J as a component
> >> implementation.
> >> 4. An attempt is made to deploy C2, which surfaces the error in J.
> >> At this point there is a serious problem as the error in J cannot
> >> be corrected without undeploying C. This could potentially be fixed
> >> by deploying a new contribution with an updated version of J but if
> >> B's import were fixed to a specific version of J's package, there
> >> is no way to guarantee the import will resolve to the contribution
> >> export containing the fixed version of J. In other words, we have
> >> now introduced a serious issue in a production environment that is
> >> entirely avoidable.
> > IMO this is a corner case that is outweighed by the benefit of
> > allowing
> > contributions with errors to be installed.
> >
>
> I think this happens more frequently than we are lead to believe. In
> any event, even if it is a corner case, it is important to allow
> runtimes the opportunity to handle it properly given the serious
> problems that may arise if it happens.
>
> >> One way to avoid this is to put the onus on users to properly
> >> structure contributions and have good test coverage. In the above
> >> scenario, A has mixed concerns and should probably be separated
> >> into distinct contributions.
> > >
> > This would be my preferred solution for the corner case.
> >
> >> A better way to avoid this is by allowing a runtime
> >> to report the error upfront and not install contribution A. This
> >> won't catch all cases (some errors depend on the state of the
> >> domain and must be detected at deployment), but it will catch a lot
> >> of them.
> > I don't think this is "better", but I also don't think SCA should
> > disallow it.
> >
> I think we just are going to need to agree to disagree here. The
> important thing is the specification should accommodate both
> approaches and allow implementors the freedom to choose the strategy
> that best suits their users.
>
> >> I think we need to take a step back and ask: why would we *require*
> >> a runtime to deploy a contribution in the above scenario? I'm happy
> >> to allow runtimes to do this but it should not be the required or
> >> mandated default behavior. In fact, most people I know would never
> >> want this to be possible in a production or staging environment.
> >> Even in development, most people I know want fail-fast behavior
> >> because they practice iterative development and have lightweight
> >> containers that boot quickly.
> > My preference for development would be different, but I think SCA
> > should allow runtimes to support either approach.
> >
> > Simon
> >
> >> Jim
> >> ---------------------------------------------------------------------
> >> To unsubscribe from this mail list, you must leave the OASIS TC that
> >> generates this mail. Follow this link to all your TCs in OASIS at:
> >> https://www.oasis-open.org/apps/org/workgroup/portal/
> >> my_workgroups.php
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this mail list, you must leave the OASIS TC that
> > generates this mail. Follow this link to all your TCs in OASIS at:
> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail. Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>
