[QFJ-727] build failure due to invalid character in source files Created: 21/Jan/13  Updated: 02/Apr/15  Resolved: 06/May/14

Status: Closed
Project: QuickFIX/J
Component/s: Build
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: amichair Assignee: Christoph John
Resolution: Fixed Votes: 0
Labels: QuickfixJ, ant
Environment:

Kubuntu 12.10, Oracle JDK 1.7.0_11

Attachments: File fixed-compilation-error-by-changing-source-file-encoding-trunk.patch     File fixed-compilation-error-by-changing-source-file-encoding.patch    

 Description   

A Fresh checkout of the QFJ_RELEASE_1_5_3 tag fails to compile (using 'ant jar' following installation instructions on website), with the error:

[javac] /home/user/dev/quickfixj-src/core/src/main/java/quickfix/CachedFileStore.java:408: error: unmappable character for encoding UTF8
[javac] * @author mratsimbazafy 29 ao�t 2008

The platform encoding on most Linux systems (and probably others too nowadays) is UTF-8, whereas the source files seem to be in ISO-8859-1 (or ISO-8859-15, whatever the devs happen to be using). A search for non-ascii characters in all source files reveals:

$ find -name "*.java" | xargs grep -P -n "[\x80-\xFF]"
./core/src/test/java/quickfix/FileLogTest.java:65: // �bcf�d��
./core/src/test/java/quickfix/AbstractMessageStoreTest.java:102: // message 111 == �bcf�d��
./core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java:119: // �bcf�d��
./core/src/test/java/quickfix/mina/message/FIXMessageEncoderTest.java:64: // �bcf�d��
./core/src/main/java/quickfix/CachedFileStore.java:408: * @author mratsimbazafy 29 ao�t 2008

The workaround would be to just delete these strings (they are all in comments), but the proper future-proof/cross-platform fix would be to convert these source files to UTF-8 encoding (e.g. using iconv) and update the build.xml files to add an encoding="UTF-8" attribute to all javac and jalopy tasks.

 Comments   
Comment by amichair [ 21/Jan/13 ]

the full fix to sources and build scripts

Comment by amichair [ 01/May/14 ]

Rebased the previous patch (which was against release 1.5.3) on top of current trunk.

Comment by amichair [ 01/May/14 ]

Any chance this will make it into trunk for the next release (or at least snapshot) sometime soon? It's pretty straightforward.

btw for the upcoming mavenization (+1!) you would use

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

(but you should still apply this patch first in order to fix the mis-encoded source files.)

Comment by Christoph John [ 02/May/14 ]

Thanks for your patch. Will integrate this into the next release.

Comment by Christoph John [ 05/May/14 ]

I have now committed the updated files. Could you please check if there are still compile errors? I could not use your patch since I am not using git so I converted the files using iconv.
Just curious: from which git repository did you get the source code?

Comment by amichair [ 05/May/14 ]

It compiles ok now as far as encoding is concerned - there is another unrelated error, but I'll open a separate issue for it.

The patch was created with 'git format-patch' from the commit in my custom branch. It's in standard diff format, so you should to be able to apply it normally with the patch utility (with the -p1 parameter, iirc) - that's how I tested it before I attached it to this issue.

I used svn2git to clone (and later rebase) the svn repository into a local mirror git repository - it uses git-svn internally with additional cleanup added. I then worked on my own local branches with fixes such as this one. I'd actually recommend migrating the project to git if you're considering it - svn2git makes it simple, and in my opinion git makes collaboration easier than svn.

Comment by Christoph John [ 06/May/14 ]

OK, thanks for testing.
Regarding git: we already have QFJ-597 and I'm planning to look at that, too.

Generated at Sun May 05 09:04:00 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.