MantisBT - VCMI
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0001288VCMIOtherpublic2013-06-01 19:442022-04-12 11:04
Reporterkrs 
Assigned ToSXX 
PrioritylowSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version0.99 
Summary0001288: Q - 春秋.h3m is not recognized as a valid map name.
Description"Q - 春秋.h3m" is not recognized as a valid map name.
Steps To ReproduceProbably renaming an existing map to "Q - 春秋.h3m" is enough.
Additional Information
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2013-06-01 19:44krsNew Issue
2013-06-01 20:18IvanNote Added: 0003619
2013-06-01 20:39TowNote Added: 0003620
2013-06-02 20:28IvanNote Added: 0003645
2013-06-03 07:33krsNote Added: 0003649
2013-06-03 07:37krsNote Edited: 0003649bug_revision_view_page.php?bugnote_id=3649#r2268
2013-06-03 09:35TowNote Added: 0003651
2013-06-03 09:39TowNote Added: 0003652
2013-06-03 11:00IvanNote Added: 0003653
2016-08-28 03:41SXXAssigned To => SXX
2016-08-28 03:41SXXStatusnew => assigned
2016-08-28 04:08SXXNote Added: 0006589
2016-08-28 20:00SXXNote Added: 0006590
2016-08-28 20:00SXXStatusassigned => resolved
2016-08-28 20:00SXXFixed in Version => 0.99
2016-08-28 20:00SXXResolutionopen => fixed
2022-04-12 11:04PovelitelStatusresolved => closed

Notes
(0003619)
Ivan   
2013-06-01 20:18   
For me it works.
This can be either Win-specific or something not related to name of file (e.g. vcmi fails to display in-game map name)
(0003620)
Tow   
2013-06-01 20:39   
I confirm that it doesn't work on Windows.
Converting path (that is natively UTF-16) to string probably breaks it.

I believe one solution would be to replace types of all filename/path variables std::string with boost::filesystem::path. That should make it work across the platforms, though would require quite a work. [And I'm not sure what'd be side-effects.]
(0003645)
Ivan   
2013-06-02 20:28   
Perhaps convert paths to utf-8? It looks that boost::filesystem supports such conversions and utf8-encoded paths works fine on my side.
(0003649)
krs   
2013-06-03 07:33   
(edited on: 2013-06-03 07:37)
Use std::wstring instead of std::string?

The reason why is working on linux could be : "Ubuntu system is already unicode aware. When I work with a char string, it is natively encoded in UTF-8 (i.e. Unicode string of chars)" http://stackoverflow.com/questions/402283/stdwstring-vs-stdstring [^]
(0003651)
Tow   
2013-06-03 09:35   
@ Ivan
> Perhaps convert paths to utf-8? It looks that boost::filesystem supports such conversions

What functoin is that? I looked through boost::filesystem docs and haven't found anything.


@ krs
Both Windows and Linux are Unicode aware. It is that Windows uses UTF-16 and Linux UTF-8.

The string/wstring are just vectors of characters, they are not meant to express encodings. Still, it is common to represent UTF-16 text as windows two-byte-chracter std::wstring and UTF-8 strings as one-byte character std::string.

But we want:
a) have as few platform-specific code as possible
b) be compatible between platforms. Eg. Linux server sends to Windows client StartInfo with non-Ascii path.

There is already wrapper using UTF-8 string on Linux and UTF-16 wstring on Windows — that is boost::filesystem. Using it instead of strings should work on all platforms and should not cause any significant costs.
The remaining problem is serializing it. I guess we could just say that path over net is in UTF-8 and make serializer do any necessary conversions.
[Plus I need to check if it actually allows on windows to open non-ascii file with plain ifstream.]
(0003652)
Tow   
2013-06-03 09:39   
Or we could try that (see the example): http://www.boost.org/doc/libs/1_53_0/libs/locale/doc/html/default_encoding_under_windows.html [^]
(0003653)
Ivan   
2013-06-03 11:00   
@ Tow

function that converts path into string accept codecvt parameter.
http://www.boost.org/doc/libs/1_53_0/libs/filesystem/doc/reference.html#string-template. [^] And as pointed out in your example codecvt can be set globally.

I also found discussion on this topic here:
http://boost.2283326.n4.nabble.com/boost-filesystem-path-as-utf-8-td4320098.html#a4322460 [^]

It seems that boost::filesystem has its own set of char convertions - I'd rather use this one instead of set from boost::locale at least for now - this is rather new library, not available in some still used boost versions.


And in case if Windows can't open utf8-encoded filename - perhaps convert it back to utf-16 before accessing file? We have relatively few cases of direct usage of fstream/fopen
(0006589)
SXX   
2016-08-28 04:08   
Probably was fixed about this pool request:
https://github.com/vcmi/vcmi/pull/156 [^]
(0006590)
SXX   
2016-08-28 20:00   
Tested. This one fixed by Zyx-2000 pull request for sure.