The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,831 other followers

Archive for the ‘XML/XSD’ Category

Offline XML Pretty Print

Posted by jpluimers on 2019/12/20

I wrote about [WayBackXML Pretty Print in Online XML Pretty Print many years ago.

That’s all fine for small XML file and on-line usage.

A while ago however, I had two cases where off-line XML pretty printing turned out to be much easier than online XML pretty printing:

  1. pretty-print many XML files in one go
  2. compare large (100 megabyte plus) XML files

So I went searching, especially for a solution that would be available for both nx based platforms (Linux/Mac OS X/etc) and Windows which got me a few options in [WayBack] unix – How to pretty print XML from the command line? – Stack Overflow.

I found [WayBack] tidy to be more practical than [WayBack] xmllint as tidy would split more lines which made it easier for  Beyond Compare (that is very much cross platform!) to spot and show the differences.

Not just easier in the sense that the (already excellent) diff engine from Beyond Compare (which really sets it apart from other differencing tools) but very much easier on the eyes as now differences where on lines below each other in stead of 1000s of character positions in one line you had to scroll through.

There is two important things to remember with tidy that both stem from its ancestry (it originates from an HTML):

  • you have to specify -xml
  • you have to specify the characterset (even though XML can specify it and without a definition, XML by default is UTF8, tidy does not know about that)

These were the command-lines I used on Windows to do the pretty printing:

tidy -xml -utf8 QCScaper.test@borland.com.cds.xml > QCScaper.test@borland.com.tidy.cds.xml
xmllint --format QCScaper.test@borland.com.cds.xml > QCScaper.test@borland.com.pretty-printed.cds.xml

Another trick is to write a small Delphi program centered around the FormatXMLData call which has been present since Delphi 2007 (see [WayBack] devnet.pdf), but documented since Delphi 2009 in [WayBack] XMLDoc.FormatXMLData Function.

Example code: [WayBack] delphi – How to reformat XML programmatically? – Stack Overflow.

An alternative is to use OmniXML: [WayBack] delphi – Nice bit of code to format an xml string – Stack Overflow

–jeroen

Related posts:

Read the rest of this entry »

Posted in Beyond Compare, Development, Power User, XML, XML/XSD | Leave a Comment »

… compare two JSON structures and pin-point … the differences – – Nicholas Ring – Google+

Posted by jpluimers on 2019/08/20

I’ve added a few WayBack/Archive.is links to the interesting comments by Zoë Peterson from Scooter Software (of Beyond Compare fame) at [WayBack] … compare two JSON structures and pin-point … the differences – – Nicholas Ring – Google+:

Beyond Compare 4 has an optional “JSON sorted” file format that uses jq to pretty print and sort JSON data before comparing it. It’s not included out of the box yet, but you can get a copy here:

If you’re interested in an actual algorithm and not just an app, I don’t have a suggestion handy, but could dig one up. Tree alignment is more complicated than sequence alignment and we did do research into it, but it was quite a few years ago and didn’t get incorporated into BC. XML alignment algorithms were being actively researched back in the aughts and they should trivially transfer to JSON.

It looks like our research mostly ended around 2002, and I wasn’t personally involved in it, so I don’t know how helpful this will be, but here’s what I have:

The general idea in the thread is that JSON – though not as formalised as XML – does have structure, so if you can normalise it, then XML ways of differencing should work.

Normalisation also means that you need to normalise any floating point, date time, escaping, quoting, etc. Maybe not for the faint of heart.

–jeroen

Posted in Beyond Compare, Development, JavaScript/ECMAScript, JSON, Power User, Scripting, Software Development, XML/XSD | Leave a Comment »

detect availability of node-set() function in xslt implementation of icecast server – Stack Overflow

Posted by jpluimers on 2019/03/07

Wow, I totally forgot I had done this xslt trick eons ago: [WayBackdetect availability of node-set() function in xslt implementation of icecast server – Stack Overflow

–jeroen

Posted in Development, Software Development, XML/XSD, XSLT | Leave a Comment »

Reminder to self: make a start for an XSD that validates Delphi dproj files

Posted by jpluimers on 2018/11/07

Below are a few interesting things from this thread [WayBack] Any idea why Delphi (At least since Seattle) does this to Dproj filesand… – Tommi Prami – Google+.

It’s likely that the WordPress system deleted parts between angle brackets because it thinks it’s invalid HTML and WordPress is PHP.

Anyhow: maybe by now I’ve found time for creating an XSD for .dproj files in addition to the one I mentioned yesterday for the .groupproj files: Validating a nested XML element with an empty namespace using XSD – Stack Overflow.

Oh and in the mean time for various Delphi versions, there is a (binary, because it uses some commercially licensed parts) [WayBackDproj changed or not changed? Normalize it! | The Art of Delphi Programming.

Files and directories I need to research:

In the mean time, QualityPortal indicates that:

Achim Kalwa:
That is a long living bug: https://quality.embarcadero.com/browse/RSP-11308
Still exists in Berlin Update 2.

Jeroen Wiert Pluimers
For more than a decade, the IDE rewriting dproj files in different order/indentation/spacing, properties appearing/disappearing in DFM files and writing DFM files despite no changes have driven me even more nuts than I was.

Walter Prins
+Jeroen Wiert Pluimers Yes. Given that .dproj is XML I’ve been almost annoyed enough to look at making some kind of pre-checkin script to force sort things to try and avoid the issue. Almost… Edit: Ah, just checked that RSP, which references this: https://github.com/joshkel/RadCli Seems someone has already done it.

Lübbe Onken
Did you try tidy_proj +Walter Prins? For me it fails with a “Unicode Encode Error” when I try to write an output file. In my case this is caused by German umlauts in the excluded bpl section descriptions. Changing the umlauts to something low-ascii allows tidy_proj to finish.

Walter Prins
+Lübbe Onken No, but I have just done so. Unicode bites again. To fix that error, change line 125 from:

print(proj.toxml())

to:

print(proj.toxml().encode(‘utf-8’))

Note: This will likely not display correctly if you allow the output to go to the console but will ensure it is correct if redirected back to file.

Edit: Ha, I was about to go report this issue ad suggest this fix, but I see you’ve already suggested essentially the same fix. ;)

Jeroen Wiert Pluimers’s profile photo
Jeroen Wiert Pluimers
+Walter Prins interesting tool! Do you know of a tool that can downgrade .dproj files so you can use them with older Delphi versions? (the opposite of what the IDE does)?

Or someone that has written more than my XSD on .dproj files? My one is so small that’s hardly a real attempt. https://wiert.me/2013/08/31/delphi-first-try-on-an-xsd-for-groupproj-files/

Walter Prins
+Jeroen Wiert Pluimers No, but interesting idea. I suppose you’re probably aware that the JEDI JCL/JVCL takes a somewhat similar (ish) related approach, in that it generates project and package files that are compatible with every desired version of Delphi from minimalist (XML) template files. Every so often I think it might be worth looking whether one can re-use the code from there easily but haven’t gotten around to it. Always something else to do first. :/

Jeroen Wiert Pluimers
+Walter Prins no I didn’t. Where should I start reading on that minimalist template approach?

Walter Prins
+Jeroen Wiert Pluimers Sorry I see I missed out the word installer in my comment: It’s the Jedi JCL/JVCL installers that uses xml templates etc.

But the point stands: I guess it should be (perhaps) possible to reuse its infrastructure for ones own projects, though as I say I keep meaning to look into this but haven’t really done so.

(BTW I’m assuming you are familiar with the JCL/JVCL and in in particular their installers that bootstrap from source code? If not then perhaps my comments may not be that useful?)

Anyway, not sure if or where there’s particular documentation about this (kind of doubt it), but if you have the JCL/JVCL installed somewhere, then first of all have a look at the “xml” folders e.g. “<jclroot>\jcl\packages\xmls” and “<jvclroot>\jvcl\packages\xml”.

These files appear to define projects and packages in a seemingly abstract/somewhat minimal way using XML. This seemingly is then used to automatically produce .dproj and .dpk files which are placed in e.g. “<jclroot>\jcl\packages\dXX” and “<jvclroot>\jvcl\packages\dXX” where dXX corresponds to a folder for each of the supported Delphi versions and compiled/used during installation.

Having just looked into this briefly a bit further as a result of this conversation: Key units here (in the case of JVCL) seems to be “<jvclroot>\devtools\PackagesGenerator\PackageGenerator.pas”, “<jvclroot>\install\JVCLInstall\PackageUtils.pas” and “<jvclroot>\devtools\common\PackageInformation.pas” (used predictably by <jvclroot>\JVCLInstall.dproj)

(Additionally, having looked at this a bit more closely, it also appears that the JCL does not actually replace all or even most of its .dproj and .dpk files after all, but ships version specific .dproj and .dpk files in most cases, though the JVCL does appear to mostly do so. )

Edit: The JCL also has this interesting sounding unit “jclroot>\source\windows\JclMsBuild.pas” which appears to be an MSBuild project file parser…

–jeroen

Posted in Continuous Integration, Delphi, Development, msbuild, Software Development, XML, XML/XSD, XSD | Leave a Comment »

Validating a nested XML element with an empty namespace using XSD – Stack Overflow

Posted by jpluimers on 2018/11/06

In a quest for making Delphi Group Project XML files (with extension .groupproj) validate with XSD, I had to find out about [WayBackValidating a nested XML element with an empty namespace using XSD – Stack Overflow.

I needed the XSD so I could import it in Delphi in order to write a good .groupproj file generator.

They are here:

And some more background posts are here:

–jeren

Posted in Delphi, Development, Software Development, XML, XML/XSD, XSD | 1 Comment »

 
%d bloggers like this: