Saturday, February 13, 2021

100,000+ "thank-you"s

At some point in the last couple of days, I passed a milestone. 

The public* packages I have made available on NuGet have now been downloaded over one hundred thousand times.

28 Packages | 101,283 Total downloads of packages

In the grand scheme of things, this isn't a large number. There are many packages with hundreds of thousands of downloads. Some of those packages get more downloads on an average day than my packages have had over several years.

So does this seem like a milestone even worth acknowledging?

I think so.

The above figure represents thousands of people (developers--they are people too) who I've been able to help save time and effort. It may be with a trivial, one-off task, or it may be something they use multiple times a day and in apps used by thousands of people. 

Download numbers like this are only relevant to the packages and the people who make them. I've known some developers over the moon because they never expected even a few hundred people to use the thing they created. In contrast, I've known other developers complain that the thing they created didn't get millions of downloads in the first few days.

This is a bigger number than I ever expected to reach but I also expect to produce many more packages in the future. Little by little, as I stick around and keep doing the work, the numbers go up. 

These tools may not have made a dent in the universe but they probably contribute to a small dimple. Maybe, over time, more dimples will add up to a dent. 

They may not be the most popular packages in the world. But that's not what I set out to create. I saw gaps where tools would be useful and produced tools to help with those situations. A small gap in the market that one person can quickly fill will rarely become massively popular, so I had set my expectations accordingly.

However, a niche product can be really useful to people who need it.

Many of these packages are now redundant (because they relate to platforms that no longer exist--the platforms aren't publicly available or supported in any way) but these packages serve as a reminder to myself (and maybe others) that I'm here for the long term. I haven't just shown up in the open-source world, made something quickly, and then gone away. It's also a reminder that small gaps can also be worth filling. Not only do they help with the things that I work on but they are also useful and helpful to others too.

I think the above figures help quantify part of my contribution to making the world a better place through open-source software.

Of course, my contributions aren't just about niche tools for underserved platforms and environments. I also contribute to much larger projects too. Some with many millions of downloads.
The combination of working on a breadth of smaller scenarios with my tools and much larger projects as part of a group of contributors, helps me become a better developer.

* There are a couple of private ones I have removed ("unlisted") because of name and dependency changes. They definitely shouldn't be used.

Tuesday, February 02, 2021

Here, I made this to search StackOverflow for your error description

Another small update to an existing Visual Studio extension.

ErrorHelper allows you to work more easily with the descriptions on the Error List.

Context menu showing options for: Copy Description, Search Description, and Open Url
You can copy the description (not just the whole line--as is the default behavior).
If the description contains a URL you can open that directly.
And you can search the internet for the description (error) message.

The alternatives to using this extension is to manually type the error message or URL. Or you can copy the whole line (including all other columns) paste it somewhere and delete the bits you don't want, or copy (again) the bit that matters and paste that into a search box somewhere else instead.

I find right-clicking and then clicking on "Search Description" much faster and easier!

In today's update (v1.7), I added the ability to set the search engine as StackOverflow.

Configuration /Options window showing search options of: Bing, Google, and StackOverflow

Yes, searching in Bing or Google quite often takes you to StackOverflow. If you find that's the case for you, this option now cuts out the middle-man.

This improvement was from a suggestion I received on Twitter. If you have any other ideas for making this (or any other extension better) then feel free to tweet me a suggestion too. 

Here, I made a thing to save time building all projects in a solution

Compilers are complicated. They don't always behave the way you'd expect.

For instance, if you have a resource that has the Copy to Output Directory setting of Copy Always this can force a whole compilation even if nothing has changed!

I've never needed this but sometimes this is the default setting for some file types. As this can lead to me waiting while projects are being unnecessarily recompiled I previously created a Visual Studio extension to detect this and warn me so I can correct offending entries and save myself time.

"Copy Always" with a line through it

Today I've updated it so that it can be used from the context menu of the Solution entry in Visual Studio's Solution Explorer.

Just a little thing but I hope it will help at least one person.


Thursday, January 07, 2021

Fallback Fonts in XAML

I have to admit that I feel foolish for not knowing about this before. Let me tell you about the power of fallback fonts in XAML so you can avoid the ignorance I lived in for so long. I'm doing this in UWP code but the same applies to WPF.

Let's say you want the following displayed (with XAML) as simply as possible.

I [filled heart] You [smiley face] [red lips] [music notes]

The "I" and the "You" are rendered with the "Segoe Script" font.

The "filled heart", "smiley face", and "music notes" are from the "Segoe MDL2 Assets" font.

The "red lips" are the "kiss mark" emoji.

Off the top of your head, how do you render this?

You might put multiple TextBlocks next to each other (probably in a Horizontally-oriented StackPanel) and assign different FontFamily values to each one.

Or, you might use multiple Runs within a single TextBlock and assign different FontFamily values to each one.

But, let me tell you there's a way to do this with a single TextBlock and no runs!

The XAML looks like this:

<TextBlock>I &#xEB52; You &#xED54; 💋 &#xE189;</TextBlock>

To get the script font I set the FontFamily to "Segoe Script" and I get this:

I [unrendered character] You [unrendered character] [red lips] [unrendered character]
Note that the red lips / kiss mark are rendered as a glyph even though they are not in the "Segoe Script" font. This is due to a built-in feature of XAML text rendering. The underlying text rendering system is doing a lot of work to try and avoid try and display all the characters. When the specified character isn't in the defined font it tries to use a value from the `ContentControlThemeFontFamily` resource. This is a list of fonts that it looks in for something to display the appropriate glyph. By default, this includes "Segoe UI" and "Segoe UI Emoji". It's the second of these that means that emojis are rendered in most places without any extra effort needed.

So, we're part of the way there by specifying a single FontFamily.

FontFamily="Segoe Script"

But what about the glyphs from the "Segoe MDL2 Assets" font?
Well, just as in CSS it's possible to specify multiple fonts/typefaces to use as a fallback, the same is possible with XAML.
Why I never thought of this before I don't know.

FontFamily="Segoe Script,Segoe MDL2 Assets"

In my head, I always thought of the HTML/CSS specification of multiple fonts as applying all or nothing to the text. If the first specified font is available use that for all the text, if it's not available use the next font, etc.

I don't know if HTML/CSS rendering is cleverer than that (please comment below, if you know) but the XAML text renderer is smarter than just using the same font for everything. It uses the specified list (family) of fonts on a character by character basis.

If it can't render a character in the first defined font, it will look to the next in the family and if none provide the character it will look to the ContentControlThemeFontFamily resource. Only if none of these fonts have a glyph for the character do you get the placeholder for a missing glyph.

This is a simplified explanation as the real solution considered different languages too. Don't ever think displaying text is simple. ;)

I discovered all this after going down a rabbit hole investigating how to avoid characters not displaying. I started by looking at intercepting what is displayed on the screen and then checking all the characters there are in the font family specified. (I didn't even consider multiple font-family values.) That lead me to investigate how to detect binding changes which led me down a side path looking at how the `x:bind` markup extension creates and reflects binding changes.

I wanted to know when any text in an app isn't rendered correctly so that the app could be updated to support the fonts/glyphs it needed so users can always see the characters they expect.

Now knowing about fallback fonts, I think a simpler solution is to make sure I include multiple fonts in the default FontFamily used within apps.

Thursday, December 31, 2020

Validate everything - why I no longer trust my supermarket

photo of a receipt highlighting entry: EASY PEELERS LSE 88.297 kg @ £1.95/kg  £172.18

The above is from the receipt when visiting a supermarket earlier in the year.
Yes, it's an entry for 88.297 KiloGrams of loose fruit at £1.95 per KG, coming to a total of £172.18!

That's almost 200 pounds and the weight of a grown adult!
It's also (to the best of my memory) more than I've ever spent in a supermarket at one time.

tldr1: Don't blindly trust ML/AI algorithms without adding extra data validation.
tldr2: Double check your receipts!

So, how did this happen?

During the lockdown, I made my once-a-week trip to the supermarket for essentials.

I used the "Smart Shop" app on my phone to scan and pack items as I shopped.
There are several possible benefits to using the app, but, to me, the biggest was reduced time with a cashier when we should be doing "social distancing."

The following is an explanation of a specific issue I've observed with the app.
I'm realistic enough to know the app won't be perfect, especially with my high standards.
There are some other issues that I regularly encounter (most times, I use the app), but I don't think they are worth exploring here. (If you're interested, these include the "scan" button disappearing and not allowing me to scan other products, and the scan button being unresponsive, leading me to press it again, which leads to the scan mode briefly being displayed before automatically closing again.)

I was doing my weekly shop, navigating my way around the store, and got to the confectionary aisle. I was choosing some treats for my children and saw something I thought my wife would like. I picked it up, scanned it. I wasn't paying attention to the app but heard the positive "beep" of a successful scan and put it in the trolley without looking at the screen.
A few minutes later, I noticed the running balance was significantly larger than what was reasonable based on my trolley. I was only shopping once a week, so spending a bit more than I had in the past, but this was way beyond anything reasonable.
I was able to scroll back through the list of everything I'd purchased and found the rogue entry. Based on the items before and after it in the list, I worked out what item was missing based on what I'd put in my trolley and where the erroneous entry had come from.
Cautiously I tried scanning the item for my wife again, and this time it showed up correctly. 
I tried removing the incorrect entry, but the app wouldn't let me without scanning it again--which was obviously impossible.
When I had gathered all my shopping and was ready to pay, I found a staff member to explain what had happened. They tried to remove the entry but required managerial approval because of the high price of the item. (Aside: why is managerial approval needed to void an item that doesn't require managerial approval to purchase?)
The item was eventually removed, and I paid for my shopping. Nobody in the store was interested in finding out more about how or why this had happened, and so I went home. (I'll save my rant on front-line customer service workers who aren't interested in escalating technical issues for another time.)

This experience weighed on my mind and left me with two big questions.
  1. How did this item end up on my list?
  2. How is this item even possible?
In reverse order.

How is this item possible?

The entry is for a product the weight of a grown adult.
The scales in the store don't support that much.
It's not physically possible to balance that many items on the scales.
How is this deemed valid?
It's a tremendous amount, so it should have raised an exception.
Why is there no default flagging or limits on the weight and total price of an individual item?
There was a check when removing the item, so why not on adding it?
Perhaps there's an implied check based on what the scales can measure, so a check wasn't deemed necessary. If the scales/printer can't print a barcode label for an item that heavy or expensive, why is it necessary to validate the value?
As we shall see, the system is based on the fact that it relies on accurate scanning, and my experience is that what is in a barcode and what the software reads can be different.

The troublesome item has what I'll refer to as a 'self-weighed barcode.' (It's a few years since I worked in a supermarket, and I don't know the terminology they use.)
These self-weighed barcodes contain two parts: an identifier for the product and the price of this particular item. This is how the correct price is charged without creating an entirely unique barcode for every weighed item.
This system is at risk of abuse, but that abuse is presumably deemed acceptable. If you've ever heard of someone weighing and printing the label for a small handful of grapes and then sticking the label on a bag containing a much larger number, you'll understand the risk. But, most people are honest, and so the amount lost by the supermarkets in this way is less than they save by not having to have everything preweighed or paying for staff to weigh them.

So, hopefully, you can see what the system thought I'd scanned, but...

How did this item end up on my list?

Or, as a bigger, more general question, how did a different item from what I'd scanned show up on my list?

Firstly, let's consider if this matters.

If it's a different product.

The store's inventory management will be out. They may think they've sold more (or less) of products. They probably allow for small discrepancies, so this may not be a problem. Whether 200 pounds of fruit counts as a small discrepancy, I can't say.

For the individual making the purchase, there are two potential consequences. Firstly they will have no proof of purchase of the product they put in their trolley/bag and so won't be able to return it or get assistance/replacement if there's a problem. The second consequence is if they are asked to verify that their shopping contents match their receipt. This may be a security check or a random check applied to all users of self-scanning systems. At best, this might require the rescanning of all items or, at worst, may be seen as an attempt at shoplifting. Good luck proving the issue was with their software if you're accused by them of shoplifting.

What if this affects an age-restricted product?
Customers may have to get their purchases checked unnecessarily. But, potentially more concerning is people buying age-restricted products without appropriate verifications being made. If my discovery has found a flaw in the supermarket's software, could this be exploited to purchase age-restricted products by people below that age? What are the legal implications of this?

If the prices are different.

If the price charged is less than it should be:
  • The store will lose money and may try and accuse you of deliberately underpaying. It might be hard for them to prove guilt, but it'll be harder for you to prove innocence. At any rate, it has the potential to be an unwelcome and stressful exchange. Again, a small discrepancy may not be worth the company's effort to change their systems to address.
  • If you're undercharged, then you gain. Bonus.

But if the price is higher than it should be...
  • The company won't mind. For them, the occasional over or undercharged item will probably cancel out.
  • Legislators might be interested, but I don't know enough about this area to say for sure.
  • For the individual, this could be a big deal. If, as in my case, it's a large difference, this could have a big impact on a person's finances. A large discrepancy is likely to be spotted, but multiple smaller differences might not be. What if you were regularly being overcharged?

How many times has this happened before, but the price difference wasn't large enough for me to notice? - It's a worrying thought.

So, what's happening when the wrong product/price show up?

Let's start by looking at the process as a whole. Here's how it's supposed to work.
  1. I scan the product with the app on my phone.
  2. The app decodes the image to identify a barcode.
  3. The data (barcode value) is sent to the server.
  4. The server responds with the product and price.
  5. Repeat for each product.
  6. At the end, the device/app sends all data to the till.
  7. I pay at the till. - Done.

The possible causes of the issue are either in the app, on the server, or in communication between them.
  • If it's a communication issue, it could be one of data corruption or a man-in-the-middle attack causing data corruption. I don't think either of these is the cause of the issue I encountered.
  • It could be that the server sent the response to the wrong device. Perhaps somewhere else, someone did scan the product that showed up on my device, and the messages got mixed up. Perhaps someone else scanned all that fruit and was only charged for some chocolate. This again seems extremely unlikely.
  • Perhaps the server was misconfigured, and it thought the barcode for the chocolate was the expensive fruit. That I scanned the chocolate again and it showed up correctly suggests this isn't the case. Of course, an invalid server configuration could have been corrected between the two times I scanned, but I think this is very unlikely.
I think the problem was with the software in the app that "reads" the barcode.
The app is running on my phone. My phone doesn't have a dedicated barcode scanner, but it does have a camera. The app uses the camera to take an image (well, it probably scans frames of video) and looks for a barcode image from which it can extract the value. (Years of building software that prints and scans barcodes have taught me a thing or two about how this works ;)

I think the "barcode scanner" software extracted the wrong value from the image/barcode, and it just happened to be something that the server considered valid.

I think this is what happened because when using the app, I regularly (it probably happens every other week) "scan" an item and receive an error message that the item is not recognized. I can "scan" the barcode again, and it goes through correctly. Is it that the barcodes are added to the system between the times I scan them? Or, is it more likely that the app sends invalid data because it "misread" the barcode?
Based on my knowledge of barcodes and image recognition, I think the app is "reading" the barcodes incorrectly. 
Don't get me wrong, I think the image recognition is impressively good. Sometimes it even reads barcodes that my knowledge of barcodes and laser scanners surprises me as I thought they'd be unreadable.

But, I know that the "barcode scanner software" isn't "reading" the barcode the same way that a laser scanner does. It uses AI (a machine-learning-based algorithm) to work out the best guess for an image's barcode value.

In this instance, I think I was "lucky" and encountered an incorrect barcode read that happened to be a value the server recognized but had a price difference enough for me to notice. This then combined with my knowledge of client-server software and barcode scanning to make an informed guess about what happened.

What could the supermarket have done to prevent this?

I have a few recommendations:
  • Continue to invest in improved accuracy of barcode "scanning" based on phone cameras.
  • Add validation for extremes of weight and price when reading values from 'self-weighed barcodes.'
  • Ensure encrypted connections between the app and server to avoid accidental corruption, modification during transmission, or responding to a different client than the sender.
  • Introduce a process for automatically logging and investigating exceptional values or errors.

What I'm doing because of this?
  • I now double-check everything I scan.
  • I verify the contents of my receipt matches what I put in my basket/trolley.
Not doing these could lead to me being overcharged and/or charged for products I didn't buy.

Bonus validation issue I found with their software for everyone who has read this far.
After several months of shopping with their phone app to scan items as I go round the store, I've discovered that there are a few aisles where the network signal is feeble. This leads to a potential issue with the app communicating with the server.

I regularly see the busy/progress spinner being displayed for longer than elsewhere in the store in these aisles. On one occasion, the delay was significantly longer than I'd encountered previously. It stopped me from scanning further items, and because I'm inclined to wonder when software misbehaves, I started counting seconds until it finished what it was doing. 
I counted for about a minute (I know I'm bad at estimating seconds, and it's not a skill I see value in spending time improving) but more concerning was that the list of products now displayed that I'd scanned two of the item I'd only scanned once. Was this another instance of the app scanning something other than I had? No, here's what I think happened.
I think the app sent the details to the server but didn't get a response. It then automatically retried (timeout and retry periods are typically either 60 or 100 seconds--both of which could have been met in this instance) and sent the request again. I think the server received both requests, but the response to the first request was not received. When the server sent the second response, it included the total number of that item that had been scanned, but this was actually the number of requests it had received, and due to the automatic retry, these weren't the same.
Having seen many, many codebases, I know it's common for developers to include automatic retry logic but not account for the server receiving a request but the response being lost.

I find it very disappointing when a large, multi-million-pound business has software that doesn't account for basic and expected connectivity scenarios.

If you're wondering which supermarket this was, why not leave a guess in the comments. ;)