MantisBT

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001288VCMIOtherpublic2013-06-01 19:442016-08-28 20:00
Reporterkrs 
Assigned ToSXX 
PrioritylowSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version0.99 
Summary0001288: Q - 春秋.h3m is not recognized as a valid map name.
Description"Q - 春秋.h3m" is not recognized as a valid map name.
Steps To ReproduceProbably renaming an existing map to "Q - 春秋.h3m" is enough.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0003619)
Ivan (developer)
2013-06-01 20:18

For me it works.
This can be either Win-specific or something not related to name of file (e.g. vcmi fails to display in-game map name)
(0003620)
Tow (developer)
2013-06-01 20:39

I confirm that it doesn't work on Windows.
Converting path (that is natively UTF-16) to string probably breaks it.

I believe one solution would be to replace types of all filename/path variables std::string with boost::filesystem::path. That should make it work across the platforms, though would require quite a work. [And I'm not sure what'd be side-effects.]
(0003645)
Ivan (developer)
2013-06-02 20:28

Perhaps convert paths to utf-8? It looks that boost::filesystem supports such conversions and utf8-encoded paths works fine on my side.
(0003649)
krs (reporter)
2013-06-03 07:33
edited on: 2013-06-03 07:37

Use std::wstring instead of std::string?

The reason why is working on linux could be : "Ubuntu system is already unicode aware. When I work with a char string, it is natively encoded in UTF-8 (i.e. Unicode string of chars)" http://stackoverflow.com/questions/402283/stdwstring-vs-stdstring [^]

(0003651)
Tow (developer)
2013-06-03 09:35

@ Ivan
> Perhaps convert paths to utf-8? It looks that boost::filesystem supports such conversions

What functoin is that? I looked through boost::filesystem docs and haven't found anything.


@ krs
Both Windows and Linux are Unicode aware. It is that Windows uses UTF-16 and Linux UTF-8.

The string/wstring are just vectors of characters, they are not meant to express encodings. Still, it is common to represent UTF-16 text as windows two-byte-chracter std::wstring and UTF-8 strings as one-byte character std::string.

But we want:
a) have as few platform-specific code as possible
b) be compatible between platforms. Eg. Linux server sends to Windows client StartInfo with non-Ascii path.

There is already wrapper using UTF-8 string on Linux and UTF-16 wstring on Windows — that is boost::filesystem. Using it instead of strings should work on all platforms and should not cause any significant costs.
The remaining problem is serializing it. I guess we could just say that path over net is in UTF-8 and make serializer do any necessary conversions.
[Plus I need to check if it actually allows on windows to open non-ascii file with plain ifstream.]
(0003652)
Tow (developer)
2013-06-03 09:39

Or we could try that (see the example): http://www.boost.org/doc/libs/1_53_0/libs/locale/doc/html/default_encoding_under_windows.html [^]
(0003653)
Ivan (developer)
2013-06-03 11:00

@ Tow

function that converts path into string accept codecvt parameter.
http://www.boost.org/doc/libs/1_53_0/libs/filesystem/doc/reference.html#string-template. [^] And as pointed out in your example codecvt can be set globally.

I also found discussion on this topic here:
http://boost.2283326.n4.nabble.com/boost-filesystem-path-as-utf-8-td4320098.html#a4322460 [^]

It seems that boost::filesystem has its own set of char convertions - I'd rather use this one instead of set from boost::locale at least for now - this is rather new library, not available in some still used boost versions.


And in case if Windows can't open utf8-encoded filename - perhaps convert it back to utf-16 before accessing file? We have relatively few cases of direct usage of fstream/fopen
(0006589)
SXX (administrator)
2016-08-28 04:08

Probably was fixed about this pool request:
https://github.com/vcmi/vcmi/pull/156 [^]
(0006590)
SXX (administrator)
2016-08-28 20:00

Tested. This one fixed by Zyx-2000 pull request for sure.

- Issue History
Date Modified Username Field Change
2013-06-01 19:44 krs New Issue
2013-06-01 20:18 Ivan Note Added: 0003619
2013-06-01 20:39 Tow Note Added: 0003620
2013-06-02 20:28 Ivan Note Added: 0003645
2013-06-03 07:33 krs Note Added: 0003649
2013-06-03 07:37 krs Note Edited: 0003649 View Revisions
2013-06-03 09:35 Tow Note Added: 0003651
2013-06-03 09:39 Tow Note Added: 0003652
2013-06-03 11:00 Ivan Note Added: 0003653
2016-08-28 03:41 SXX Assigned To => SXX
2016-08-28 03:41 SXX Status new => assigned
2016-08-28 04:08 SXX Note Added: 0006589
2016-08-28 20:00 SXX Note Added: 0006590
2016-08-28 20:00 SXX Status assigned => resolved
2016-08-28 20:00 SXX Fixed in Version => 0.99
2016-08-28 20:00 SXX Resolution open => fixed

Site | Forums | Wiki | Slack | GitHub


Copyright © 2000 - 2018 MantisBT Team
Hosting provided by DigitalOcean