QR code scanning via megapixels

Hardware Librem 5
1

Howdy,
I spent some time this morning making a postprocessing script that one can use with megapixels to process and use qr codes with little to no effort for the user.

I recognize that megapixels does detect qr codes, but it doesn’t seem to be able to do anything with them. The copy to clipboard never works for me.

How to use it?

  1. Install the postprocess script (see below)
  2. Open up megapixels and point your phone at a qr code.
  3. If a QR code is detected a blue box will appear around the qr code (this is via megapixel’s own processing not this script)
  4. Take a picture of the qr code with that blue box around it. (bottom middle circular button)
  5. Wait for the picture to be processed(You’ll see a spinning thing in the bottom left). This may take a few seconds
  6. A dialog box should pop up saying a qr code was found in the provided image. Depending on the qr code, it will ask if you want to see the contents or perform an action based on the contents.
    NOTE: if you say yes to performing an action related to the qr code, it will delete the image, as presumably it’s not needed anymore. If you guys disagree with this functionality let me know.

I’m a bit surprised that I couldn’t find an instance of someone else doing this as it was trivially easy to write. Here’s installation instructions and the code:

In theory this should work on both the pinephone and the librem 5. I’ve only tested it on the librem 5.

Edit: some things to note:

  1. The qr code libraries don’t seem to recognize frame qr or micro qr codes, so those won’t work.
  2. If the code can be read but not understood, it will just open the text editor and show you the text of the qr code.
  3. I haven’t figured out a way to launch chatty saying “Hey try to start a sms conversation with this phone number”. So it just opens up sms qr codes in the text editor sadly… Same thing with wifi qr codes.
  4. Please feel free to contribute to or suggest improvements. They are appreciated!

Note, if there’s a better way that everyone’s been using, please do let me know.

13 Likes
2

Great work. I hope the different methods of launching stuff work out.

About deleting the code after use: I probably would want to keep the codes for some limited time “just in case” and have the codes stored to a separate folder. At least for 15mins or an hour, after which they could be automatically removed. If it could be made into a preference that allows for easy selection of what and when this happens, I think that would be optimal (delete immediately or store for x amount of time - 15mins, 2h, 24h, 1week, forever/no deletion). And I’d probably prefer to store the content/text of the code, rather than the image (append new line to a text file maybe? with date/time stamp - easier to come back to).

Did you get coordinates/location/address to start a map? Perhaps a preference setting or dialog about what to start if such are encountered? Although, an html link to (for instance) openstreetmap probably already works.

2 Likes
3

So effectively a log? I think that would be pretty easy to do, though some of the qr codes are multi line

Yes, On my phone it will launch puremaps asking what to do with it(set it as a point, destination starting point, etc). Note that for launching things it uses xdg-open, and thus should open things using whatever the system default application is. Thus if someone wants to change that all they should need to do is change the default application.

I’d rather not write a separate one and let the system handle it. However, I do note that even though there is no setting for a default map application it seems to open pure maps anyways using xdg-open. I’m looking into that now.

1 Like
4

Very very interesting. Congratulations on the job
:+1:

5

Unfortunately that has been relatively unreliable if not impossible of late. It was working previously.

So this will only work for me if using the selfie camera and, absent a mirror, that’s going to be in the too hard basket.

It may help to use --raw on the zbarimg command. That will avoid having to strip out the symbology.

1 Like
6

I’ll confess that I haven’t seen that issue on my phone. I do know that when I delete the postprocess.sh script from my home directory it no longer can save picture for some reason(I had a postprocess script from syncing my homedir from mobian on the pinephone).

I’ll look at --raw option, thanks!

7

Sadly the main camera hasn’t been working for a couple of weeks for me (while the selfie camera does work). I did have a working script before that. (My script is a lot more basic than yours. The sole goal of my script was QR code for COVID check-in.)

8

Gotcha, I hope purism fixes that soon!

1 Like
9

Based on your suggestion, I’m considering writing up a gui that:

  1. Keeps a history of the recently taken QR codes for a user specified amount of time
  2. Allows saving and naming qr codes for later use that it offers a list of.
  3. Provides the ability to add/modify actions for how to handle different types of qr codes

After that create a separate program that’s intended to offer the ability to turn various types of data into qr codes via plugins. Examples of plugins I’m thinking of:

  1. Offer your currently configured wifi as a wifi qr code
  2. Allow you to share redisplay your saved qr codes as qr codes
  3. Ability to write your own custom qr codes and save those

Thoughts?

1 Like
10

I think you should be, and will be, praised for your effort, is what I think.

I’m wondering if it would be beneficial for usability to have it work so that the reading ability will be in both, as now via Megapixels and can be separate As in, basic reader ability and the the other stuff as well via GUI.

The history option sound neat. From usability perspective, it would be nice to have a “re-do last code” button vs. “open list & copypaste”, if that’s not too much. On the other hand, there may be a need to edit the text (perhaps a typo in the original) before use/launch, as well as a need to turn that edited text to a new qr - close to what you have already in the plan.

If you’re looking for features to add, how about an optional visible/readable text below or besides the code. For instructions or content description or for and advert (maybe an optional default text) etc. It would save from copypasting the qr to an editor. Definitely something for later stages of development.

About the other plugins, I’m thinking those in terms of an easy way to transfer info (especially in a way that is pretty private and safe, if it’s via the phone camera from the screen of a friend or yourself). Especially strings that need to be correct (yes, like wifi pw etc.). How about a method of relaying the info for a quick access to shared screen or setting up shared secrets or relaying GPG? Custom codes sound good - could your app help set it up as a process, where a custom prefix could launch a designated app with the data or some other action(s)? One common one would be sharing contact details but there are several options: the plain text (with a nice layout - same row or stacked) or one of the virtual calling card standards.

And what about adding an encryption to the text? A bit overboard, but a simple pw mechanism might be useful for some. For a more advanced usecase, the pw could be replaced with location - as in: vicinity needed to open the qr (to prevent/inconvenience global use).

I think there will be limits that will be met and your GUI could help tell how much text/data can be crammed into the qr vs. size, or whether it would be smarter to think making the qr a link to the large text/data/file/object.

I hope these keep you busy. There are probably suporterts for a sleek, minimalist version as well.

11

If that means that the Librem 5 is configured as a hotspot and it is displaying a QR code so that clients can connect to the Librem 5 as hotspot then I wouldn’t bother with implementing that because the Librem 5 can either already do that or will be able to do that.

On the other hand if that means that the Librem 5 is configured as a WiFi client and it is displaying a QR code so that other would-be WiFi clients can connect to the same WAP that the Librem 5 is connected to then beware the security issues but OK.

If you are going in the direction of creating QR codes then I would suppose that for starters you should have GUI to solicit the information needed to create a valid QR code for each of the types of QR code that you support scanning of. (That would be without plugins and definitely before getting to custom QR codes.)

In other words, right now you have decode of: mailto, mecard, geo, tel, smsto, http, https

So I am suggesting that you add encode of those options.

And also add decode and encode of wifi:

Business card QR codes come in two overall types, web links (URLs) and real business cards, and within the latter type they come in various formats. For example, I just picked one web site at random and their business card decodes as BEGIN:VCARD …

Aside: Web link business cards look like the latest privacy scam from the web. I’m not suggesting that a site offering such a service is actually a scam but I can see where that would be going … (if anyone wants to pursue that comment, please fork the topic)

There are no real standards for most of this.

See also an earlier discussion starting at Camera development progress

That earlier discussion mentions sms:

12

PS I would change the match of http* to separate matches of http and https because you don’t really know what the future might bring for other schemes whose name starts with http, and any given user of your script may not be sufficiently knowledgeable to know whether such a future scheme is safe.

13

Note thattmy current vision is that there would be a separate app from megapixels for the history and whatnot. For the history:

  1. It would just display a list of your previous qr codes giving the type of qr code, and data (perhaps truncated) in text format.
  2. taping on the item would replay it. Effectively asking you what you want to do with it. One option being launching something with it (like the postprocess script does) (no copying and pasting required)

Didn’t think about the editing though. I’ll need to think about that. Will need to be careful to not make that too easy though lest people mess up their codes. Perhaps only the ‘saved codes’?

Indeed, those sounds like good plugins. I’m going to start small though(very few plugins to start), and my hope is to make plugins easy enough to write to encourage people to write their own personal one, or contribute them to let others use them.

Current thought process for plugins is just a directory that would house scripts (each script being a plugin) and I would just take any stdout and qr codify it (assuming it returned 0). Hopefully that would make it really easy for others to write their own.

That’s part of the idea behind the “Provides the ability to add/modify actions for how to handle different types of qr codes”

Yes, that’s the idea behind that one. The security issues I can see with that are:

  1. I may need to use sudo to get the information. If that’s the case, it will need to be very targeted, and not do anything else.
  2. There’s the risk that the code could be seen and scanned by others nearby.
    Is there annother threat that you’re concerned about?

I’m not sure I follow, but I do agree that documentation is needed. Largely at present I’m trying to pull in the existing standards.

You’re absolutely right. I need to be a little less lazy there.

1 Like
14

Not that I can think of. Those were the basic points that I had in mind. Any time you are messing around with security-relevant info, it is a good idea to consider the “what ifs”.

You will need sudo and that makes it all the more fraught (even though needing sudo is obviously a good thing).

Out of the box, user purism is set up with sudo access without entering a password (so sudo will work regardless of whether it is a good idea). However I have changed my phone to require a password (and that could in theory break some scripts).

Sorry for the stream of consciousness. The basic point is that, as a starting point, whatever you can decode you should be able to encode, where “encode” is referring to that separate program that you were talking about to create QR codes.

So for example if you have code to take an smsto: QR code image, decode the QR code into the embodied string, then pick out the destination phone number and message (then actually offer to send that text message) then …

the reverse process would be code to

  • solicit a phone number and message (form with two entry fields)
  • convert that into a valid smsto: string
  • convert that into a QR code image

By the way, are your delimiters correct for smsto: handling in the decode? From what I’ve seen, the format is

smsto:optional-phone-number:optional-message

and the optional-message may include the colon character (so programmer beware).

So smsto:: is valid (when scanned would launch the program to send a text but neither the phone number nor the message would be prepopulated) and smsto::: is valid (phone number not prepopulated and message prepopulated with a single colon character). Yes, these are extreme cases.

1 Like
15

One other comment: Per RFC 1738, schemes are supposed to be recognised without regard to case. So you ought to be standardising the case for comparison purposes but you must not standardise the case of the entire URL. So the easiest approach may be to extract the scheme (assumption is that everything starts with a scheme followed by a colon character), downcase it, and then compare only against lower case. Or, even easier, use the nocasematch shell option (untested by me).

I am definitely seeing QR codes where the scheme is in unexpected case so this is not a theoretical comment.

2 Likes
16

Ok I believe all of the mentioned issues should be fixed now.

1 Like
17

How do you have a qr code scanner when the rest of us don’t have a camera app? last I checked with the latest developments on Librem5 software?

18

megapixels is the current camera app. Purism has slowly been getting the camera to work better and better. Granted a few phones have issues like irvinewade’s.

1 Like
19

Purism didn’t make megapixels, but it is available for install from their software app. It’s been used on the pinephone for some time now for picture taking.

20

To be exact, Megapixels used on the Librem 5 is a heavily modified, forked version that uses different V4L API, removes features that are not relevant with our sensors and adds some that are not relevant to PinePhone’s ones.

Currently the selfie cam should reliably work for everyone (at least on Dogwood batch forward, not sure about the earlier ones), while the rear cam depends on your luck. On one of my phones it works perfectly (see https://social.librem.one/tags/shotonlibrem5 :wink: ), on other one it’s capricious, and on another one it’s basically unusable. This should improve with further work on the driver.

6 Likes