Wow, I totally forgot I had done this xslt trick eons ago: [WayBack] detect availability of node-set() function in xslt implementation of icecast server – Stack Overflow
–jeroen
Posted by jpluimers on 2019/03/07
Wow, I totally forgot I had done this xslt trick eons ago: [WayBack] detect availability of node-set() function in xslt implementation of icecast server – Stack Overflow
–jeroen
Posted in Development, Software Development, XML/XSD, XSLT | Leave a Comment »
Posted by jpluimers on 2018/11/07
Below are a few interesting things from this thread [WayBack] Any idea why Delphi (At least since Seattle) does this to Dproj filesand… – Tommi Prami – Google+.
It’s likely that the WordPress system deleted parts between angle brackets because it thinks it’s invalid HTML and WordPress is PHP.
Anyhow: maybe by now I’ve found time for creating an XSD for .dproj files in addition to the one I mentioned yesterday for the .groupproj files: Validating a nested XML element with an empty namespace using XSD – Stack Overflow.
Oh and in the mean time for various Delphi versions, there is a (binary, because it uses some commercially licensed parts) [WayBack] Dproj changed or not changed? Normalize it! | The Art of Delphi Programming.
Files and directories I need to research:
In the mean time, QualityPortal indicates that:
Achim Kalwa:
That is a long living bug: https://quality.embarcadero.com/browse/RSP-11308
Still exists in Berlin Update 2.Jeroen Wiert Pluimers
For more than a decade, the IDE rewriting dproj files in different order/indentation/spacing, properties appearing/disappearing in DFM files and writing DFM files despite no changes have driven me even more nuts than I was.Walter Prins
+Jeroen Wiert Pluimers Yes. Given that .dproj is XML I’ve been almost annoyed enough to look at making some kind of pre-checkin script to force sort things to try and avoid the issue. Almost… Edit: Ah, just checked that RSP, which references this: https://github.com/joshkel/RadCli Seems someone has already done it.Lübbe Onken
Did you try tidy_proj +Walter Prins? For me it fails with a “Unicode Encode Error” when I try to write an output file. In my case this is caused by German umlauts in the excluded bpl section descriptions. Changing the umlauts to something low-ascii allows tidy_proj to finish.Walter Prins
+Lübbe Onken No, but I have just done so. Unicode bites again. To fix that error, change line 125 from:print(proj.toxml())
to:
print(proj.toxml().encode(‘utf-8’))
Note: This will likely not display correctly if you allow the output to go to the console but will ensure it is correct if redirected back to file.
Edit: Ha, I was about to go report this issue ad suggest this fix, but I see you’ve already suggested essentially the same fix. ;)
Jeroen Wiert Pluimers’s profile photo
Jeroen Wiert Pluimers
+Walter Prins interesting tool! Do you know of a tool that can downgrade .dproj files so you can use them with older Delphi versions? (the opposite of what the IDE does)?Or someone that has written more than my XSD on .dproj files? My one is so small that’s hardly a real attempt. https://wiert.me/2013/08/31/delphi-first-try-on-an-xsd-for-groupproj-files/
Walter Prins
+Jeroen Wiert Pluimers No, but interesting idea. I suppose you’re probably aware that the JEDI JCL/JVCL takes a somewhat similar (ish) related approach, in that it generates project and package files that are compatible with every desired version of Delphi from minimalist (XML) template files. Every so often I think it might be worth looking whether one can re-use the code from there easily but haven’t gotten around to it. Always something else to do first. :/Jeroen Wiert Pluimers
+Walter Prins no I didn’t. Where should I start reading on that minimalist template approach?Walter Prins
+Jeroen Wiert Pluimers Sorry I see I missed out the word installer in my comment: It’s the Jedi JCL/JVCL installers that uses xml templates etc.But the point stands: I guess it should be (perhaps) possible to reuse its infrastructure for ones own projects, though as I say I keep meaning to look into this but haven’t really done so.
(BTW I’m assuming you are familiar with the JCL/JVCL and in in particular their installers that bootstrap from source code? If not then perhaps my comments may not be that useful?)
Anyway, not sure if or where there’s particular documentation about this (kind of doubt it), but if you have the JCL/JVCL installed somewhere, then first of all have a look at the “xml” folders e.g. “<jclroot>\jcl\packages\xmls” and “<jvclroot>\jvcl\packages\xml”.
These files appear to define projects and packages in a seemingly abstract/somewhat minimal way using XML. This seemingly is then used to automatically produce .dproj and .dpk files which are placed in e.g. “<jclroot>\jcl\packages\dXX” and “<jvclroot>\jvcl\packages\dXX” where dXX corresponds to a folder for each of the supported Delphi versions and compiled/used during installation.
Having just looked into this briefly a bit further as a result of this conversation: Key units here (in the case of JVCL) seems to be “<jvclroot>\devtools\PackagesGenerator\PackageGenerator.pas”, “<jvclroot>\install\JVCLInstall\PackageUtils.pas” and “<jvclroot>\devtools\common\PackageInformation.pas” (used predictably by <jvclroot>\JVCLInstall.dproj)
(Additionally, having looked at this a bit more closely, it also appears that the JCL does not actually replace all or even most of its .dproj and .dpk files after all, but ships version specific .dproj and .dpk files in most cases, though the JVCL does appear to mostly do so. )
Edit: The JCL also has this interesting sounding unit “jclroot>\source\windows\JclMsBuild.pas” which appears to be an MSBuild project file parser…
–jeroen
Posted in Conference Topics, Conferences, Continuous Integration, Delphi, Development, Event, msbuild, Software Development, XML, XML/XSD, XSD | Leave a Comment »
Posted by jpluimers on 2018/11/06
In a quest for making Delphi Group Project XML files (with extension .groupproj) validate with XSD, I had to find out about [WayBack] Validating a nested XML element with an empty namespace using XSD – Stack Overflow.
I needed the XSD so I could import it in Delphi in order to write a good .groupproj file generator.
They are here:
And some more background posts are here:
–jeren
Posted in Delphi, Development, Software Development, XML, XML/XSD, XSD | 1 Comment »
Posted by jpluimers on 2018/09/11
Schema Central, by [WayBack] Datypic, Inc. (Priscilla Walmsley) is to golden that I quote the whole index page below.
The index refers to XML vocabularies. Each vocabulary page links the corresponding XSD pages. Each XSD page lists documentation, elements, etc. Each element page then documents the element, shows where it is referenced from.
Welcome to Schema Central, an interactive tool to traverse and search XML schemas. Please choose your XML vocabulary of interest to get started. Don’t see the XML vocabulary you use? Ask us to add it.
Core XML Technologies
- XML Schema: 1.0 and 1.1
- XML 1.0
- ISO Schematron
- XLink
- XProc
- XSLT: 1.0 and 2.0
- XInclude
- XForms: 1.0 and 1.1
- XML Catalogs 1.1
Presentation-Oriented Standards
- XHTML 1.0: (strict), (transitional) and (frameset)
- XSL-FO 1.1
- CALS Table Model (Oasis XML Exchange Table Model)
- MathML 2.0 and 3.0
- SVG 1.1
Office/Narrative Document Standards
Business Document Standards
- OAGIS® 9.4.1, 10
- UBL 2.0, 2.1 and 2.2
- StratML
- FpML
- 5.5: Confirmation, Pretrade, Recordkeeping, Reporting, Transparency
- 5.0: Confirmation and Reporting
- XBRL 2.1
NIEM-related Standards
- NIEM: 2.0, 2.1, 3.0, 3.1, 3.2 and 4.0
- LEXS: 3.1.4 and 4.0
- ULEX: 1.0 and 2.0
- N-DEx IA (Incident/Arrest) 2.1
- N-DEx IBP2 (Incarceration/Booking Probation/Parole) 2.1
- ANSI/NIST-ITL 2011
Miscellaneous
Web Services Technologies
- WSDL: 1.1 and 2.0
- SOAP: 1.1 and 1.2
- WS-Addressing 1.0
- WS-AtomicTransaction 1.1/1.2
- WS-BPEL 2.0
- WS-BusinessActivity 1.2
- WS-CDL 1.0
- WS-Coordination 1.2
- WS-Discovery 1.0 and 1.1
- WS-Eventing
- WS-Notification 1.3 (includes WS-BaseNotification, WS-BrokeredNotification and WS-Topics)
- WS-Policy 1.5
- WS-ReliableMessaging 2005 and 1.1/1.2
- WSRF (Resource Framework) 1.2 (includes WS-BaseFaults, WS-Resource, WS-ResourceProperties, WS-ResourceLifetime and WS-ServiceGroup)
Security-Related Technologies
- IC Trusted Data Format (Public version; includes IC-ARH, IC-EDH, IC-ID, IC-TDF, IC-ISM, IC-NTK)
- SAML 2.0
- WS-Security
- WS-SecurityPolicy 1.2 and 1.3
- WS-SecureConversation 1.4
- WS-Trust 1.3 and 1.4
- XML Encryption
- XML Signature
- XACML 2.0 and 3.0
Site developed and hosted by Datypic, Inc.
Please report errors or comments about this site to contrib@functx.com
[WayBack]
If I ever need a long term XML/XSD/XSLT/XQuery hire, this is the “about” information I need:
Datypic provides consulting services and training, specializing in XML, content management and SOA/Web Services architecture and development. We are experts in XML-related technologies such as XML Schema, XSLT and XQuery, and have extensive experience with software development and implementation.
We participate in consulting projects ranging from one day to many months, anywhere in the world. We can arrange to work remotely or at your site, whichever you prefer.
For more information, please read about our services or our company, or contact us as pwalmsley@datypic.com.
Recent news from Priscilla Walmsley
- I recently updated my book XQuery to match the final XQuery 3.1 recommendation. You can read more about it or order it from O’Reilly or Amazon.
- I recently taught a course at the XML Summer School on Refactoring XSLT. Slides are available as a PDF.
- More information about my recent books and articles is available on this site.
–jeroen
Posted in Development, Software Development, XML, XML/XSD, XPath, XSD, XSLT | Leave a Comment »
Posted by jpluimers on 2018/09/06
A valid SOAP message with <urn:timeStamp>9999-11-31T23:59:59.9999999</urn:timeStamp> in a xs:dateTime field return '9999-12-31 23:59:59,1000' is not a valid date and time from a Delphi application with this SOAP response:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>SOAP-ENV:Server</faultcode>
<faultstring>'9999-12-31 23:59:59,1000' is not a valid date and time</faultstring>
<faultactor/>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
The reason is this exception:
exception class EConvertError with message ''9999-12-31 23:59:59,1000' is not a valid date and time'.
This is from a .NET based test case passing in timeStamp = DateTime.MaxValuewhich is handled perfectly fine by other SOAP web services tested.
I know about different resolutions of time stamps, but would never expect the 999.9999 milliseconds to be rounded up to 1000 as it is always safer to truncated away from an upper limit.
A test using Soap UI [WayBack] with this parameter finally worked (max 3 digits second fraction):
<urn:timeStamp>9999-12-31T23:59:59.999</urn:timeStamp>
The true origin of problem is in this method in the Soap.XSBuiltIns unit which has been unchanged since at least Delphi 7:
function TXSBaseTime.GetMilliSecond: Word; begin Result := Round(FractionalSeconds*1000); end;
The problem exposed itself because as of Delphi XE6 the core of function TXSBaseCustomDateTime.GetAsDateTime piece was changed from
Result := EncodeDateTime(Year, Month, Day, Hour, Minute, Second, 0);
to
Result := EncodeDateTime(Year, Month, Day, Hour, Minute, Second, Millisecond);
A combination of lack of test cases and understanding XML specifications failed to reveal this bug.
The standards specify (among others):
'.' s+ (if present) represents the fractional seconds;All ·minimally conforming· processors ·must· support year values with a minimum of 4 digits (i.e., YYYY) and a minimum fractional second precision of milliseconds or three decimal digits (i.e. s.sss). However, ·minimally conforming· processors ·may· set an application-defined limit on the maximum number of digits they are prepared to support in these two cases, in which case that application-defined maximum number ·must· be clearly documented.s -- represents a digit used in the time element "second". The two digits in a ss format can have values from 0 to 60. In the formats described in this specification the whole number of seconds ·may· be followed by decimal seconds to an arbitrary level of precision. This is represented in the picture by "ss.sss". A value of 60 or more is allowed only in the case of leap seconds.Strictly speaking, a value of 60 or more is not sensible unless the month and day could represent March 31, June 30, September 30, or December 31 in UTC. Because the leap second is added or subtracted as the last second of the day in UTC time, the long (or short) minute could occur at other times in local time. In cases where the leap second is used with an inappropriate month and day it, and any fractional seconds, should considered as added or subtracted from the following minute.The reproduction is quite simple:
Posted in .NET, C#, Conference Topics, Conferences, Delphi, Development, Event, SOAP/WebServices, Software Development, XML, XML/XSD | Leave a Comment »
Posted by jpluimers on 2018/09/05
Sometimes, the Delphi WSDL importer imports fine, but the generated code does not accept test cases sent by other tools.
Below are some links for messages and comment fragments that I want to investigate further.
// Cannot unwrap:
// - Input element wrapper name does not match operation's name
// - The output part is not a complex type// CODEGEN: Generating message contract since the operation prematchChanged is neither RPC nor document wrapped.I have included the .NET message, because my experience is that searching on those gives more accurate results for something that could be broken in more than one environment.
Based on those:
ReplyAction="*" seems to be causing a lot of questions:
Some on-line tools prefer the WSDL to be in one document, but a lot of WSDL documents use import and or include features, so here are some links on that too:
wsdl:import, but now also supports wsdl:include)Bruneau Babet correctly informed me that – though Delphi SOAP clients support both document literal and RPC encoded – Delphi SOAP servers cannot support document literal, as they can only support RPC encoded. Through that I found
The Win32/native SOAP server is currently on the Deprecated list and is classified as “Other Consideration: Need alternative.” If you want a document/literal server, you should build your SOAP server using Delphi for .NET (RAD Studio), which supports the SOAP specifications that are supported by the .NET framework.
The Win32 SOAP server support generates the older RPC|Encoded style WebServices. If you want a document/literal service or a service of any other WS-I compliant style, you should build your SOAP server using Delphi Prism, which supports the SOAP specifications that are supported by the .NET framework, including WS-I compliant styles.
Unable to create a SOAP Server Application (WebService) with [WayBack] WS-I Basic Profile 1.1 compliance.
As far as I know, with delphi we can developed RPC/Encoded web services but there is no way to create Document/Literal service.
There are some companies that continue to use SOAP for information transfer, SAP one of them and we had to use C # to develop the application server.
With C# only [WayBack]
[SoapDocumentMethod]or [WayBack][SoapRpcMethod]is assigned to the Web Method.
Back on those days, the big plan was to move everything Delphi to the .NET platform which supports both document literal and RPC encoded.
All in all, document literal has been on the radar with the Delphi R&D team since at least 2009, and nothing has been done.
References:
import, but not include)WS-I Basic Profile v1.1 provides guidance for using SOAP 1.1, WSDL 1.1, and UDDI 2.0.
WS-I Basic Profile v2.0 provides guidance for using SOAP 1.2, WSDL 1.1, UDDI 2.0, WS-Addressing, and MTOM.
wsdl:import is for a WSDL importing another WSDL
wsdl:definitions element and it has namespace and location attributesxsd:import is for a schema in a WSDL importing an XSD having a different namespace (many sites incorrectly refer to this as “WSDL import element”)
xsd:schemaelement and it has id, namespace and schemaLocation attributesxsd:include is for a schema in a WSDL including an XSD having the same namespace
xsd:schema element and it has id and schemaLocation attributesR1126 An INSTANCE MUST use a “500 Internal Server Error” HTTP status code if the response message is a SOAP Fault.env:Server, a statuscode 400 is to be used, but WS-I mandates 500.I looks like a wsdl message request part entries need to be named parameters for some tooling to correctly infer document/literal in a wrapped way. Some links for further research on this:
Note how the word “wrapped” does not appear. What IBM in their document is calling “document/literal/wrapped” is simply “document/literal”, that happens to use a single message part, that happens to have a name derived from the name of the service, and that happens to refer to an element, and which happens to contain both the parameters to the operation.
When you are surely running SOAP over HTTP, you can use this small class to raise exceptions which automatically get translated into SOAP Faults having the right return code using a trick I bumped into a few years ago from [WayBack] web services – Accessing the original TWebRequest object in a Delphi SOAP Server – Stack Overflow:
unit SoapFaultWithHttpCodeExceptionUnit;
interface
uses
System.SysUtils;
type
ESoapFaultWithHttpCodeException = class(Exception)
strict private
FHttpStatusCode: Integer;
public
constructor Create(const AHttpStatusCode: Integer);
property HttpStatusCode: Integer read FHttpStatusCode;
end;
implementation
uses
Winapi.WinHTTP,
Soap.WebBrokerSOAP,
Web.HTTPApp,
IdCustomHTTPServer;
constructor ESoapFaultWithHttpCodeException.Create(const AHttpStatusCode: Integer);
var
IdHTTPResponseInfo: TIdHTTPResponseInfo;
ReasonString: string;
WebDispatcher: IWebDispatcherAccess;
begin
IdHTTPResponseInfo := TIdHTTPResponseInfo.Create(nil, nil, nil);
try
FHttpStatusCode := AHttpStatusCode;
IdHTTPResponseInfo.ResponseNo := AHttpStatusCode;
ReasonString := Format('%d: %s', [AHttpStatusCode, IdHTTPResponseInfo.ResponseText]);
inherited Create(ReasonString);
// https://stackoverflow.com/questions/10405227/accessing-the-original-twebrequest-object-in-a-delphi-soap-server
if Supports(GetSOAPWebModule, IWebDispatcherAccess, WebDispatcher) then
begin
WebDispatcher.Response.StatusCode := HTTP_STATUS_SERVER_ERROR;
WebDispatcher.Response.ReasonString := ReasonString;
end
finally
IdHTTPResponseInfo.Free;
end;
end;
end.
—jeroen
Posted in .NET, C#, Conference Topics, Conferences, Delphi, Development, Event, SOAP/WebServices, Software Development, XML/XSD | Leave a Comment »
Posted by jpluimers on 2017/11/02
Quoted in full because even 2.5 years later, it’s just too funny:
- Python: What if everything was a dict?
- Java: What if everything was an object?
- JavaScript: What if everything was a dict *and* an object?
- C: What if everything was a pointer?
- APL: What if everything was an array?
- Tcl: What if everything was a string?
- Prolog: What if everything was a term?
- LISP: What if everything was a pair?
- Scheme: What if everything was a function?
- Haskell: What if everything was a monad?
- Assembly: What if everything was a register?
- Coq: What if everything was a type/proposition?
- COBOL: WHAT IF EVERYTHING WAS UPPERCASE?
- C#: What if everything was like Java, but different?
- Ruby: What if everything was monkey patched?
- Pascal: BEGIN What if everything was structured? END
- C++: What if we added everything to the language?
- C++11: What if we forgot to stop adding stuff?
- Rust: What if garbage collection didn’t exist?
- Go: What if we tried designing C a second time?
- Perl: What if shell, sed, and awk were one language?
- Perl6: What if we took the joke too far?
- PHP: What if we wanted to make SQL injection easier?
- VB: What if we wanted to allow anyone to program?
- VB.NET: What if we wanted to stop them again?
- Forth: What if everything was a stack?
- ColorForth: What if the stack was green?
- PostScript: What if everything was printed at 600dpi?
- XSLT: What if everything was an XML element?
- Make: What if everything was a dependency?
- m4: What if everything was incomprehensibly quoted?
- Scala: What if Haskell ran on the JVM?
- Clojure: What if LISP ran on the JVM?
- Lua: What if game developers got tired of C++?
- Mathematica: What if Stephen Wolfram invented everything?
- Malbolge: What if there is no god?
–jeroen
Posted in .NET, APL, Assembly Language, BASIC, C, C#, C++, COBOL, Development, EPS/PostScript, Fun, Go (golang), Java, Java Platform, JavaScript/ECMAScript, LISP, Makefile, Pascal, Perl, PHP, Python, Quotes, Ruby, Rust, Scala, Scripting, Smalltalk, Software Development, T-Shirt quotes, TCL, Turbo Prolog, VB.NET, Visual BASIC, XML/XSD, XSLT | Leave a Comment »
Posted by jpluimers on 2017/07/25
I’ve been using cURL but always had a feeling not to its potential basically because the cURL man page [WayBack] is both massive and lacks concrete useful practical examples.
For instance, I knew about the --header and --verbose options (I always use verbose names even though shorter -H and -v exist) to pass a specific header and get verbose output, but the man page basic examples like this by Tader:
curl --header --verbose "X-MyHeader: 123" www.google.comsource: How to send a header using a HTTP request through a curl call? – Stack Overflow [WayBack]
There are some more examples at bropages.org/curl but they’re hardly organised or documented.
So I was really glad I found the below answer [WayBack] by Amith Koujalgi to web services – HTTP POST and GET using cURL in Linux – Stack Overflow.
But first note that recent versions (around 7.22 or higher) of cURL now need to combine the --silent and --show-error (or in short -sS) parameters to suppress progress but show errors: linux – How do I get cURL to not show the progress bar? – Stack Overflow [WayBack]
Posted in *nix, Communications Development, cURL, Delphi, Development, HTTP, https, Internet protocol suite, JavaScript/ECMAScript, JSON, Power User, REST, Scripting, Security, Software Development, TCP, TLS, XML, XML/XSD | 1 Comment »
Posted by jpluimers on 2017/06/29
XML processing never gets easy. So here are some links that helped me:
–jeroen
Posted in Development, Software Development, XML, XML/XSD, XPath | Leave a Comment »
Posted by jpluimers on 2017/06/21
A while ago, I had to fix some stuff in an application that would write – using a binary mechanism – UTF-8 and UTF-16 strings (part of it XML in various flavours) to the same byte stream without converting between the two encodings.
Some links that helped me investigate what was wrong, choose what encoding to use for storage and fix it:
TEncoding.UTF8.GetBytes: utf 8 – String to byte array in UTF-8? – Stack OverflowSysUtils.ByteLength() is not the nicest function, so use only for a quick-fix Delphi Unicode String Length in Bytes – Stack OverflowAnsiString, UnicodeString et al: String Types (Delphi) – RAD Studio–jeroen
Posted in Delphi, Delphi 10 Seattle, Delphi 10.1 Berlin (BigBen), Delphi XE8, Development, Encoding, Software Development, UTF-16, UTF-8, UTF16, UTF8, XML, XML/XSD | 3 Comments »