Unicode in USB Product string not supported.

Hi,

it is valid for USB iProduct to use Unicode, for instance I am using a . However, this is a problem for PCSC, I get:

00001539 hotplug_libudev.c:184:HPReadBundleValues() Error parsing /usr/lib/pcsc/drivers/ifd-ccid.bundle/Contents/Info.plist

for my entry

<string>SoloKeys Solo 🐝</string>

in /etc/libccid_Info.plist.

Maybe it is a bad idea to use Unicode as other libraries may also not support Unicode...

But: Should this library support Unicode?

I don't see any reason to refuse UTF-8. But I have never tested this so a (stupid) bug may be present.

I will have a look.

The problem is that the Info.plist parser does not accept UTF-8 characters. See https://salsa.debian.org/rousseau/PCSC/-/blob/master/src/tokenparser.l#L75

Adding UTF-8 support to flex is not easy. See https://stackoverflow.com/questions/921648/how-to-make-a-flex-lexical-scanner-to-read-utf-8-characters-input

Maybe a drop-in replacement parser can be used? I don't know. Why does the parser need to interpret the characters vs. simply passing them through? I don't quite understand.

By the way, the descriptor strings use UTF-16LE, not UTF-8, for some historical Windows reason.

I may have found a solution to accept UTF-8 characters in the Info.plist. I need to make more tests.

Your remark about descriptor strings is interesting. The libccid driver uses libusb_get_string_descriptor_ascii() to get those strings.

• Does your device use non-ASCII characters in a descriptor string?
• If yes what is the output of the parse command? See https://ccid.apdu.fr/#CCID_compliant

Hi Ludovic, thank you for your interest in this topic!

I am using the unicode character 'HONEYBEE' (U+1F41D) in the product string, it looks like this:

The full iProduct string is Solo 🐝, this is then written into the descriptor in UTF-16-LE (not! UTF-8) encoding as follows: https://github.com/mvirkkunen/usb-device/blob/master/src/descriptor.rs#L200-L203

Output of dmesg (and lsusb, and usbview) is displayed correctly (note that VID/PID are planned but not definitive):

[220572.580166] usb 1-2.1: New USB device found, idVendor=1209, idProduct=beee, bcdDevice= 1.23
[220572.580170] usb 1-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[220572.580172] usb 1-2.1: Product: Solo 🐝
[220572.580174] usb 1-2.1: Manufacturer: SoloKeys

The output of your parse command replaces the two non-ASCII bytes with ?? as follows:

Parsing USB bus/device: 1209:BEEE (bus 1, device 56)
 idVendor:  0x1209  iManufacturer: SoloKeys
 idProduct: 0xBEEE  iProduct: Solo ??
  Found a CCID/ICCD device at interface 0

Interestingly, if I modify the Info.plist file to contain <string>SoloKeys Solo Bee</string>, then I can communicate via pcscd without problems. So it seems the correspondence between iProduct and plist is not strict, and the plist string entry is only used for logging output, e.g. in

00000004 readerfactory.c:1075:RFInitializeReader() Attempting startup of SoloKeys Solo Bee (20/20) 01 00 using /usr/lib/pcsc/drivers/ifd-ccid.bundle/Contents/Linux/libccid.so

So, strictly speaking, it is possible to use current CCID/PCSC with Unicode in iProduct already today :)

In the Info.plist only the ifdVendorID and ifdProductID fields are used to identify a smart card reader.

The field ifdFriendlyName is used to set the PC/SC name. It can be anything (except Unicode for now).

If you could help me debug parse that would be fine. I think I need to use libusb_get_string_descriptor() instead of libusb_get_string_descriptor_ascii(). What should I use for the langid parameter?

It is possible for USB devices to offer string descriptors in multiple languages.

In practice, most devices use English (US), which is 0x0409. But you cannot rely on this, instead you need to query the device for supported languages and pick the first, cf. https://www.beyondlogic.org/usbnutshell/usb5.shtml#StringDescriptors

As far as I can see, libusb does not expose this query outside libusb_get_string_descriptor_ascii, so you need to copy its code: https://github.com/libusb/libusb/blob/26611eaa494ed9e077b5b0e1f999f5ae377de958/libusb/descriptor.c#L1063-L1086

And I also would need a function to convert from UTF-16-LE into UTF-8. Grrr!

I don't think that is worth the effort. You have the FIRST reader to use non-ASCII

I will just continue adding support of UTF-8 in the Info.plist.

Yes, this makes sense. parse is just a helper utility, so it can do "lossy" descriptor to ASCII conversion. Whoever adds a reader that has Unicode descriptors (me ) can modify its output manually, before adding entries in readers/.

I understand Info.plist is generated at build time from the readers/ directory (all files in UTF-8, no UTF-16-LE), so I think that's the part that needs fixing, and you have a plan for this. Correct?

mentioned in commit 434f75ec

Fixed. I should make a new release of pcsc-lite "soon".

closed

mentioned in commit 8d19fb00

Great to hear @rousseau!

Given the relatively low frequency of releases, I'd love to get our device in for the next release in preliminary form (who knows when distributions pick up releases...). There are two problems:

1. I need to actually apply for 1209:BEEE with pid.codes; for this I need to open source the code (which I'll do "soon")
2. The ATR we use is preliminary as I'm not sure yet what all we have to include (currently I'm just sending [0x3b, 0x8c,0x80,0x01]

As I understand it, the ATRs included in this repository are informational only, so 2. should not be a blocker. Similarly I strongly expect 1. to be granted since we meet all the requirements (hw/fw open source).

Would it make sense for me to send as PR, and if so, under "should work" or "unsupported"? I'm also happy to use Solo Bee instead of Solo 🐝 if it helps, but given all your work...

My goal with this is to avoid our beta test users having to manually patch and compile pcsclite when they receive our tokens in the near future.

BTW, I already have assigned 1209:CC1D from pid.codes (http://pid.codes/1209/CC1D/), perhaps it could make sense to also/alternatively add this together with the minimal ATR above as "generic open source reader", for other developers to use before they decided on a specific VID/PID pair + ATR?

Your ATR is incorrect. See https://smartcard-atr.apdu.fr/parse?ATR=3B8C8001

If you want to have your device in the CCID "should work" list please follow https://ccid.apdu.fr/#CCID_compliant

I did not know this special vendor ID 1209. I already have the device 0x2702 in my list https://ccid.apdu.fr/ccid/shouldwork.html#0x12090x2702

Yes, I would be happy to be in the beta testers team :-)

https://ludovicrousseau.blogspot.com/2020/05/unicode-characters-in-reader-name.html