RE: [wsrp] WSRP url parameters and URL Encoding questions

see 
below,

 

regards,

Andre

  
-----Original Message-----
From: Rich Thompson 
  [mailto:]
Sent: 31 March 2003 15:06
To: 
  
Subject: RE: [wsrp] WSRP url parameters 
  and URL Encoding questions

  

The way the spec reads now, it is the 
  Consumer's responsibility to return to the Producer whatever was specified. So 
  for the two cases: 
 1. Consumer 
  rewriting: Consumer's task to encode/decode parameters for round trip through 
  user agent so that it can resupply them correctly to the Producer. 
  
[Andre Kramer] producer must also not include any 
  "&" in parameter values for consumer writing.

  
 
 2. Producer writing: Producer's job to encode/decode/interpret 
  properly when parameters make round trip through user agent (Consumer just 
  reads from the activated URL and places the value into the appropriate 
  field). 
[Andre Kramer] The consumer's Web Server is 
  likely to do a "URL decode" as the URL params come in to it. Best to forward 
  unencoded to the producer, I think.

  
 
The discussion below sounds to me like some testing may be in order and 
  then a section of the primer can discuss guidance in this area. 
 
[Andre Kramer] The question 
  is how much up front work we put in to make inter-op possible (even if common 
  Web Servers URL decode and recognize the '/' url path separator) and 
  probable.

  
 

  
 Not 
  sure we need to be modifying the spec to say what all the issues might be, but 
  do agree with the general comment that started this thread that the spec say 
  that all parameters need to be URL encoded by the 
  Producer. 

[Andre Kramer] If not the 
  spec, we need a document with some ability to make interoperability 
  conformance statements (and a process to allow us to agree on them). Something 
  that does not have to be frozen at 1.0. An Implementer's best 
  practice?

  
 
Rich Thompson 
  

  
    
    

      
      Andre Kramer 
        <> 
        
03/31/2003 04:01 AM 

              
        To:     
            
        cc:       
          
      
          Subject:        RE: [wsrp] WSRP url 
        parameters and URL Encoding 
  questions

Some 
  <ak> comments below.

-----Original Message-----
From: Eilon 
  Reshef [mailto:]
Sent: 28 March 2003 
  22:07
To: 
Subject: RE: [wsrp] WSRP url 
  parameters and URL Encoding questions

Rethinking about it - in the 
  end of the day, URL encoding can be done by two
actors:

<ak> A 
  special case is wsrp-url where the consumer must be able to get 
  the
(producer's) actual parameter value. The other parameters hold data 
  that is
opaque to the consumer or data that does not really need to contain 
  any %aa.
</ak>

When the Producer writes the full URLs using 
  the provided templates, the
question is what happens when the Producer 
  encodes a navigational state of
e.g., a%20b. I believe (and may be wrong 
  here) is that the standard states
that the Producer should expect to 
  receive the navigational state "a b" in
the getMarkup() call. If the 
  Producer so wishes, it may encode information
further but that should be 
  transparent to the protocol. (Eventually portlets
will use an auxiliary 
  function provided by the Producer to encode the
various parameters and the 
  Consumer is oblivious to the content, so it's
mostly a Producer 
  decision.)

When the Consumer writes the full URLS, it parses the 
  "original" URL
provided by the Producer to find the WSRP-specific 
  parameters. The Producer
must thus encode illegal characters to ensure that 
  the Consumer can parse
the URL, and to that end the encoding mechanism must 
  be defined. (It is then
up to the Consumer to rewrite the URL in a way that 
  will "survive" across
the interaction with the user - the Consumer may 
  decide to re-escape
elements or to encode them in other ways, but this 
  decision is made solely
by the Consumer and does not seem to affect 
  WSRP.)

<ak> The consecutive '/'s problem can be addressed by the 
  consumer in the
second case but we should have the same rule for "//" for 
  both schemes. So
that the producer should be able to treat incoming param 
  values in the same
way for both schemes. <ak>

I believe 
  (and may be wrong here again) that this is the situation captured
in the 
  spec.

<ak> We could clarify editorially (as Rich suggests) by 
  stating the need to
URL encode all (producer's) parameter values (that may 
  contain special
chars) and that this layer of encoding is removed by the 
  consumer or Web in
both schemes (on the way back in to the producer). 
  </ak>

Now, it seems to me that if a Producer generates a Consumer 
  URL that has
wsrp-navigationalState=x%aa%aay, where %aa is the URL encoding 
  for the
character '/', and the Consumer incorrectly passes back a 
  navigation state
of 'x/y', then this is an incorrect Consumer behavior. If 
  that behavior is
done before the Consumer gets control (e.g., by the Web 
  server or by a
gateway) then we may need to prohibit the %aa as a 
  parameter, for otherwise
we can expect to see many erroneous 
  Consumers.

<ak> We only need to prohibit "//", "///" etc (i.e. 
  runs of %2f in (singly)
encoded form). However, this formalizes a 
  work-around and still means the
consumer must still carefully handle "/" 
  (only a minor implementation issue
for some consumers).

We could ban 
  all %xy from parameter values (i.e. no % chars at all). Very
draconian, but 
  may prevent interoperability problems down the road.

We could just ban 
  "/" and give no advice (apart from clarify producer encode
required but no 
  producer decode required) [as you suggest.]

We could also point out 
  that a double url encode will hide any "/" (and
other special chars but %) 
  from all but the producer (who must then also do
a decode). But if we don't 
  require this, then the consumer must still be
careful about parameter value 
  contents (e.g. it may care about all '/'s) and
wsrp-url remains a problem. 
  

Requiring a double encode may be a step to far. So, I agree a ban on 
  "/" is
simplest. A required double encode (by the producer for both URL 
  schemes so
we have one set of rules) and a single producer decode would 
  give some added
protection but may be more than we should be 
  specifying.

With a '/' ban, we still need to be clear on wsrp-url. We 
  could just allow
'/' in wsrp-url proxy urls, as the consumer can have 
  simple heuristics to
deal with the "://" issue (the "//" is redundant 
  anyway!). 
</ak>

Eilon

-----Original 
  Message-----
From: Subbu Allamaraju [mailto:] 
Sent: 
  Friday, March 28, 2003 8:37 PM
To: 
Subject: 
  Re: [wsrp] WSRP url parameters and URL Encoding questions

Eilon 
  Reshef wrote:
> A somewhat different approach would be to disallow 
  "special" 
> characters
> in the portlet-provided values, e.g., 
  navigational state, etc.
>  
> This would mean that a portlet 
  can only use, say, URL-allowed 
> characters
> (alphanumeric+), 
  but not /, #, :, ?, =, ;, %, etc. (the full list is 
> documented in the 
  corresponding RFC).

I prefer encoding. Applications/portlets may use 
  special characters 
(say, "Smith & Co") in state, and it is safer to 
  encode those characters 
 instead of disallowing such 
  strings.

> This will make it easy for the Consumer to process (no 
  special 
> handling,
> no dependency on Web server), and will also 
  make it easy for portlets 
> and Producers (easy rules, the portlet can 
  always do one level of 
> escaping and replace the % with, say a dash 
  (-).

Is this not somewhat similar to requiring portlets to URL-encode 
  
parameters? So, why not requre the consumer do this? This would be less 
  
restritive for developers.

Subbu

>  
> 
  Escaping seems to be one of the more common sources for 
> 
  incompatibility
> and it may just be safer to sidestep rather than spend 
  a lot of time to 
> ensure that compatibility across Producers Consumers 
  and Web servers.
>  
> My two cents,
> Eilon
> 
  
>     -----Original Message-----
>     
  *From:* Andre Kramer [mailto:]
>     
  *Sent:* Friday, March 28, 2003 7:41 PM
>     *To:* 
  
>     *Subject:* RE: [wsrp] WSRP url 
  parameters and URL Encoding 
> questions
> 
>     
  Rich,
>      
>     Unfortunately, the 
  collapsing of // to / also happens within
>     parameters not 
  just to the separators, e.g. the navigationalState
>     
  parameter value written into the template by the producer is "a//b"
> 
      but is modified to "a/b" following normal url path processing 
  
> rules.
>      
>     The whole 
  path is subjected to this collapsing. The consumer can
>     
  protect itself from this for its own parameter separators as you
> 
      suggest (and even have special smarts for wsrp-url values), but 
  the
>     producer should not need to special case any "//" in 
  its URL
>     parameter values (wsrp-navigationalState and 
  wsrp-interactionState
>     are the only difficult 
  parameters). That is why I'm suggesting doing
>     a double 
  encoding. We could instead suggest no consecutive '/'s
>     
  allowed in navigationalState and interactionState but that seems
> 
      very arbitrary.
>      
>   
    regards,
>     Andre
> 
>     
      -----Original Message-----
>         
  *From:* Rich Thompson [mailto:]
>       
    *Sent:* 28 March 2003 16:42
>         *To:* 
  
>         *Subject:* Re: 
  [wsrp] WSRP url parameters and URL Encoding 
> questions
> 
  
> 
>         It would be a good editorial 
  clarification to comment that all
>         
  parameter values should be URL encoded since they will appear in
> 
          the URL activated by an End-User interaction. Does 
  anyone object
>         to adding such a 
  comment?
> 
>         As to the particular 
  question about collapsing // into /, the
>         
  Consumer can easily prevent this by using a construct such as /_
> 
          to separate parameters (or "/ns_" before the 
  navigational state
>         if so desired). I think 
  we can leave it the Consumer's
>         
  responsibility to properly construct its templates.
> 
>   
        Rich Thompson
> 
> 
>     
                      
   *Andre Kramer <>*
> 
> 
          03/28/2003 10:48 AM
> 
>     
                        
        
>               
    To:        
>   
                cc:       
   
>                 
  Subject:        [wsrp] WSRP url parameters and URL
> 
          Encoding questions
> 
> 
> 
  
> 
>         In 10.2.1.1.4.1 we advice 
  "wsrp-url" values to be URL Encoded.
> 
>       
    However, we are silent on the remaining consumer rewriting
> 
          tokens and their
>       
    producer URL writing counterparts. But the obvious thing to do
> 
          is to URL
>         
  encode them all (i.e. if {wsrp-navigationalState} contains an
>   
        "&" or a "/"
>         etc 
  then URL Encode it. Or encode it anyway just to be safe).
> 
> 
          Having URL Encoded all parameters, for producer 
  template URL
>         activation ,
>   
        the web server may even help out and do the URL decode 
  for 
> you.
> 
>         Furthermore, in 
  order to support method GET, URL templates must
>       
    avoid query
>         strings. One strategy 
  is to use a path ("/") instead, but I have
>         
  found that
>         (after the above helpful URL 
  decode) some Web Servers will
>         replace any 
  "//"
>         with a "/"!
> 
>   
        [A valid transformation for file paths as, e.g. file path 
  a///b
>         == file path
>     
      a/b, but not great if one is encoding data as a path. We 
  should
>         not force
>     
      consumers to use "#" or ";" instead of "?", as these also 
  have
>         issues.]
> 
> 
> 
          Obviously this corrupts any (URL template) 
  parameters that contain
>         consecutive back 
  slashes, and we can not expect producers to
>       
    know what URL
>         structure the 
  consumer is using for it's templates. Both the
>       
    decode and
>         collapsing consecutive 
  "/"s seem valid things for the Web server
>         
  to do but
>         they are interacting with our 
  method=GET work around. What could
>         we 
  do?
> 
> 
>         The simplest 
  solution seems to be to *double* URL Encode values
when
>   
        replacing a  {wsp-someparameter} in templates. By 
  double encode
>         I mean
>     
      {wasp-paramValue} =
>         
  HttpUtility.UrlEncode(HttpUtility.UrlEncode(RawParamValue)) or
>   
        {wsrp-paramValue} =
>         
  URLEncoder.encode(URLEncoder.encode(rawParamValue));
> 
>   
        [A single URL encode should be enough for consumer 
  rewriting.
>         The consumer
>   
        can apply a second encode on re-writing (if 
  required).]
> 
>         This double encode 
  does seem onerous at first, but has the
>         
  advantage of
>         being always safe and 
  independent of template schemes and
>         
  usesMethodGet (as
>         well as constant for a 
  Web Server environment & avoids us
>         
  inventing a new
>         encoding scheme).
> 
  
>         What do people think?
> 
> 
          regards,
>         
  Andre
> 
>