Preservation of whitespace for strings?

[salk31]

Hi,

Sorry in advance if this is a bad issue but I'm really out of my depth as very new to SOAP and SUDS.

In the code I'm working on, carriage returns are getting changed into new lines while going via SUDS. I've not seen this problem in the Java based code.

I might be looking at the wrong spec but https://www.w3.org/TR/xmlschema-2/#rf-whiteSpace suggests the default (which I think it is in our XSD) is "preserve". This makes sense to me and would obviously be helpful in my case.

Is this just a limitation of SUDS that it uses standard XML whitespace handling?

I had a quick look at the current code and I'd be up for attempting a pull request (even if just to change the default behaviour for string) but I might
have totally the wrong end of the stick.

I've got a monkey patch that works for my narrow case but it would be nice if it was in core SUDS.

Thanks

Sam

[phillbaker]

Hi Sam, thanks for opening an issue. Can you please include an example showing the white space conversion you’re seeing? A test case would be a great way to do that if the WSDL isn’t public. Please also share the patch you mentioned, I’d be happy to take a look and evaluate it. Note that we’d want to avoid changing existing default behavior, so we may need an option or to gate it otherwise.

…

On Thu, Aug 21, 2025 at 7:53 AM salk31 ***@***.***> wrote: *salk31* created an issue (suds-community/suds#116) <#116> Hi, Sorry in advance if this is a bad issue but I'm really out of my depth as very new to SOAP and SUDS. In the code I'm working on, carriage returns are getting changed into new lines while going via SUDS. I've not seen this problem in the Java based code. I might be looking at the wrong spec but https://www.w3.org/TR/xmlschema-2/#rf-whiteSpace suggests the default (which I think it is in our XSD) is "preserve". This makes sense to me and would obviously be helpful in my case. Is this just a limitation of SUDS that it uses standard XML whitespace handling? I had a quick look at the current code and I'd be up for attempting a pull request (even if just to change the default behaviour for string) but I might have totally the wrong end of the stick. I've got a monkey patch that works for my narrow case but it would be nice if it was in core SUDS. Thanks Sam — Reply to this email directly, view it on GitHub <#116>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXCKMFKEUJLSW6TDJHQKT3OWXLHAVCNFSM6AAAAACEOOFVTOVHI2DSMVQWIX3LMV43ASLTON2WKOZTGM2DCNBUGYZTQMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[salk31]

Hi Phill,

A very tentative/minimal unit test (although the case I really care about) master...salk31:suds:whitespace-handling. A more general case I think would need to worry about trailing/leading whitespace too.

I'm a bit puzzled by no handling of numeric entity references, is this just a missing feature or not relevant to SOAP?

Here is our WSDL and XSD ICAT.zip I can't see any explicit whitespace configuration.

btw I've not got a patch written, I was just saying that I'm up for attempting to write something. Although I'm very new to SOAP/SUDS and relatively new to Python.

I take your point about not changing behaviour on people without control.

Thanks
Sam

[phillbaker]

I've got a monkey patch that works for my narrow case but it would be

nice if it was in core SUDS. Can you share this monkey patch?

…

On Tue, Aug 26, 2025 at 6:10 AM salk31 ***@***.***> wrote: *salk31* left a comment (suds-community/suds#116) <#116 (comment)> Hi Phill, A very tentative/minimal unit test (although the case I really care about) master...salk31:suds:whitespace-handling <master...salk31:suds:whitespace-handling>. A more general case I think would need to worry about trailing/leading whitespace too. I'm a bit puzzled by no handling of numeric entity references, is this just a missing feature or not relevant to SOAP? Here is our WSDL and XSD ICAT.zip <https://github.com/user-attachments/files/21985720/ICAT.zip> I can't see any explicit whitespace configuration. btw I've not got a patch written, I was just saying that I'm up for attempting to write something. Although I'm very new to SOAP/SUDS and relatively new to Python. I take your point about not changing behaviour on people without control. Thanks Sam — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXCKOYIK762JJDSHQVITD3PQXC7AVCNFSM6AAAAACEOOFVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRTGUZDMNBZG4> . You are receiving this because you commented.Message ID: ***@***.***>

[salk31]

Of course:

from suds.sax.enc import Encoder
Encoder.encodings = (
    ("&(?!(amp|lt|gt|quot|apos);)", "&"),
    ("<", "&lt;"),
    (">", "&gt;"),
    ('"', "&quot;"),
    ("'", "&apos;"),
    ("\r", "&#13;"))
Encoder.special = ("&", "<", ">", '"', "'", "\r")

A lot of copy n paste with the tuples being immutable...

[salk31]

Sorry, re-read my original message. I meant preferring a fix in the core rather than my dodgy patch.

[phillbaker]

Yup, understood you’re suggesting a fix in the suds library. Can you include a few lines of code to reproduce the issue using that wsdl? What is an example message where you’re seeing the problem? I took a brief look at the code and I see a path where trailing/leading white space would be trimmed, but I didn’t see an obvious place where carriage returns are translated or removed.

…

On Tue, Aug 26, 2025 at 3:58 PM salk31 ***@***.***> wrote: *salk31* left a comment (suds-community/suds#116) <#116 (comment)> Sorry, re-read my original message. I meant preferring a fix in the core rather than my dodgy patch. — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXCKNGND5RRL33B7EZLWT3PS373AVCNFSM6AAAAACEOOFVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRVGU2TCOBSHA> . You are receiving this because you commented.Message ID: ***@***.***>

[salk31]

Thanks, on the replacement of carriage returns (after 25+ years working with XML I only just found this out) apparently XML is opinionated on whitespace https://www.w3.org/TR/REC-xml/#sec-line-ends so it is really the receiving XML parser that is replacing the carriage returns (as it should)

It might take me a while to come up with a minimal example. I've never looked at WSDL before... I'll look at the other automated tests in SUDS.

[phillbaker]

Any update on this? Is suds working as expected?

…

On Fri, Aug 29, 2025 at 1:03 AM salk31 ***@***.***> wrote: *salk31* left a comment (suds-community/suds#116) <#116 (comment)> Thanks, on the removal of carriage returns (after 25+ years working with XML I only just found this out) apparently XML is opinionated on whitespace https://www.w3.org/TR/REC-xml/#sec-line-ends so it is really the receiving XML parser that is removing the carriage returns (as it should) It might take me a while to come up with a minimal example. I've never looked at WSDL before... I'll look at the other automated tests in SUDS. — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXCKMU2X7SDITRSISZJDL3P7NKFAVCNFSM6AAAAACEOOFVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZVG4ZTENRZG4> . You are receiving this because you commented.Message ID: ***@***.***>

[salk31]

Sorry, I've been tied up with support issues at work. I'm still pretty convinced SUDS isn't following the spec but I've not got a minimal example.

I was thinking I'd try and extend one of the WSDL unit tests then parse the result with a standard parser in Python. Would that help? I don't really see why it has to be our (quite big WSDL).

One of the support issues is that the system is trying to send through ASCII char 3 through the pipeline. I think it is a related issue. SOAP should allow any character to be transmitted (with options)? OK if I broaden this issue?

Sorry for dragging it out, I thought I was entering a quiet period of work so I could focus on this.

[salk31]

A lot of cargo cult but does this help https://github.com/salk31/suds/blob/whitespace-handling/tests/test_string_encoding.py ?

I still need to test \0x03 on the Java SOAP soap and how JAXB handles it.

Sorry if that was not what you were after.

[phillbaker]

Thanks, I’ll take a look. Just to confirm, that test fails for you?

…

On Sat, Sep 6, 2025 at 1:33 PM salk31 ***@***.***> wrote: *salk31* left a comment (suds-community/suds#116) <#116 (comment)> A lot of cargo cult but does this help https://github.com/salk31/suds/blob/whitespace-handling/tests/test_string_encoding.py ? I still need to test \0x03 on the Java SOAP soap and how JAXB handles it. Sorry if that was not what you were after. — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXCKLKDGMZVZL2RVF22L33RMLHRAVCNFSM6AAAAACEOOFVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENRSG4ZDMMRTGM> . You are receiving this because you commented.Message ID: ***@***.***>

[salk31]

Yes, seems consistent with the XML spec and what I'm seeing in our system. CR -> NL and CR NL -> NL

[salk31]

I was getting a bit paranoid about my chain of logic. I dumped the response from payara/glassfish to see what it does:

<?xml version="1.0" encoding="UTF-8"?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://icatproject.org/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <S:Body>
      <ns2:searchResponse>
         <return xsi:type="ns2:datafile">
            <createId>db/root</createId>
            <createTime>2025-09-08T16:28:58.299+01:00</createTime>
            <id>3970918099</id>
            <modId>db/root</modId>
            <modTime>2025-09-08T16:28:58.299+01:00</modTime>
            <datafileCreateTime>1971-01-06T02:02:03.004+01:00</datafileCreateTime>
            <datafileModTime>1971-01-06T02:02:03.004+01:00</datafileModTime>
            <description>unknown</description>
            <fileSize>3</fileSize>
            <location>/dlstest/i02/data/2025/mx1174-1/bad_char_2_&#13;.txt</location>
            <name>bad_char_2_&#13;.txt</name>
         </return>
      </ns2:searchResponse>
   </S:Body>
</S:Envelope>

It didn't escape NL. I've yet to check 0x03.