xml:base and fragment identifiers

If I have an xml document with xml:base set to some uri,
then use a cross reference / link to (what used to be) another id
in the same document using a fragment identifier, would you expect the 
target to be
calculated relative to this same document, or relative to the xml:base value?

Would it make any difference if it were an xhtml document?

Is it wholly application dependent?

TIA, DaveP.