Friday, June 18, 2021

I don't think you should only test manually

In response to my post When it's Ok to break the build some people thought I was trying to argue against using CI/CD tooling to test changes.

This isn't the case.

Only running tests manually and/or on your main development machine can be slow and a waste of resources.

Creating a draft PR to test against a large test suite and only raise to a full PR when everything has been confirmed as being ok is a good way of working.

Pushing local changes to a private branch with a CI/CD process running tests against it can be a good way of working.

Either of the above is also a good way of incorporating the execution of extensive test suites or tests that involve multiple environments or varied configurations that would be impractical to set up or maintain on each developer's machine.


My objections and concerns come when a full (non-draft) PR is raised before tests have been run.



BTW. I'm currently planning a book all about testing. It will focus more on the concepts than the details, but I think I can bring some new ideas to the subject. (Hopefully, I'll do a better job at avoiding confusion than I did in that other post.) If you want to know more, be sure to sign up for my (very occasional) newsletter. 

Thursday, June 17, 2021

When it's Ok to break the build

For some teams, and some developers, breaking the build is a terrible thing that should be avoided at all costs. (Not just because you may have to buy everyone donuts if you do.)

Dunkin'Donuts donut box


I used to be fine with anyone breaking the build (and not just because I like donuts.)

This came with some qualifications:

  • You shouldn't do it deliberately.
  • You should be making small, frequent changes that are less likely to break anything.
  • You should fix the build ASAP if you do. (You don't go home having broken the build as you'll be blocking other people.)
But, all in all, I'd not sweat it if someone broke the build. I'd rather they were doing that because they were shipping code and not sitting on massive changes or holding back because they were worried about breaking it.

For the most part, this approach has worked well for many years.

But recently, I've encountered behavior that's making me reconsider this idea.


The behavior I've seen is developers using the CI process as a way to validate their code.

I've seen attitudes like:

"If it fails, I'll find out when the CI process tries to build it as part of the PR automation."

"I won't bother running the tests myself as the CI process will do it for me."

"If the CI process fails, it will create a new issue, and if I fix that, it looks like I'm closing more issues."


These thoughts scare me:

  • They indicate a lack of professionalism and pride in the work being done.
    (Why wouldn't you check that it works before saying you're finished?)

  • They indicate a lack of respect for the value of testing.
    (Why wouldn't you check that there are no unintended consequences of the changes you've made by running the tests? - Ok, It may take a while, but surely it's better to be safe than sorry. If running them takes a long time, isn't it better to know about problems sooner? And especially to know about them before you get engrossed in whatever the next thing is.)

  • They suggest you can work the system to look more valuable and productive, and presumably, the organization rewards this.
    (This sounds like you've found a way to make your life simpler at a cost to the product, the company, and the team/your peers. That doesn't sound healthy. It sounds like your manipulating the data to appear to be closing more bugs when actually you're the cause of many of them.)


Don't be like that.
Make an effort to ensure that your code works, doesn't break anything else or have unintended side effects, and that you're helping your team to all be as productive as possible.


So, when is it ok to break the build?

If it's an accident that you attempted to avoid, and will fix it quickly.


When is it not Ok to break the build?

There are many reasons, but my current bug-bear is that it's not ok to break the build if you do it because of laziness.


Saturday, June 05, 2021

Is this TDD? (When you can't do: Red-Green-Refactor)

I've been thinking a lot about testing recently.

Specifically, I've been thinking about using TDD and legacy code. (Yes, I've got Michael Feathers's book and will need to reread it. This is probably covered there.)

So, I've just received a bug report that I decide to work on and fix:

  1. I created some tests that recreate the issue. Running them verified that they reproduced the issue (they failed) so my tests were RED.
  2. I then changed the code so the tests passed. Now my tests were all GREEN.
  3. Then I tidied up the code a bit more. (I REFACTORED the code.)
  4. Finally, I ran the tests again to make sure they all still pass. They did.

  5. But then I thought of another edge case that wasn't currently tested but probably should be.
  6. So, I create a test. And, it passes!
Oh, no! It didn't go Red first, so I can't be doing "proper TDD"™.

I'm happy that the code does correctly handle (and document) this test case but how should I test it "properly"?

  • Change the code so the test fails? and then change it back?
  • Change the test so it fails? and then change it back?
  • Just leave it?
  • Something else?

What would you do?



BTW. I'm currently planning a book all about testing. It will focus more on the concepts than the details but I think I can bring some new ideas to the subject. (Despite what you may think by me asking this question.) If you want to know more, be sure to sign up for my (very occasional) newsletter. 

Friday, May 28, 2021

Free t-shirts for XAML developers

 Today I got to celebrate the Rapid XAML Toolkit officially reaching 1500 installs. 

partial screenshot of the VS marketplace showing 1500 user installs for Rapid XAML Toolkit
(It has more users than this but that discussion can be for another day.)

I was also reminded I still have this box of t-shirts under my desk since the start of last year and obviously haven't been able to give them away at the conferences I was originally planning to.

Bok of blue t-shirts with the Rapid XAML logo

So, let's have a give-away.

I have t-shirts and a desire to improve and promote my tools.

You have an interest in obtaining a t-shirt and ideas that may help me improve the tool.

How's about this:

To enter the draw for a t-shirt:

Then on June 10th 2021, I'll randomly pick some people who've created issues and send them t-shirts.


Obviously, the goal here is for me to gain some useful new feature requests or suggestions. I'm also interested in how people are using the toolkit and so if you don't have any ideas for new features it's also useful for me to know how it's being used (telemetry is deliberately very limited) so please share what you've done with it or how it's helped.


I couldn't possibly say if leaving a positive (5⭐) review in the marketplace will help your chances of winning, but it definitely won't hurt ;)


Conditions:

  1. This is for fun and to help me tidy my "office"
  2. Any created issues that don't contain useful (as I determine it) details will be excluded from the draw.
  3. The number of winners will depend on the number of entries and the number of t-shirts I have (I don't know right now) 
  4. Maximum of one prize per person but feel free to make multiple feature requests.
  5. The selection of winners will be as "random" as I can easily manage.
  6. Winners will be notified once and will have a limited amount of time to claim their prize. (I'm not going to chase winners indefinitely.)
  7. I'll pay postage. Import duty, taxes, or anything once I've posted it are on you.
  8. Don't like how this giveaway is being run? Tough.
  9. If this is successful I may do this again. No promises.

Sunday, May 16, 2021

Rapid XAML Toolkit v0.12 - Now with extra Xamarin.Forms related goodness

After a longer than desired period of time, version 0.12 of the Rapid XAML Toolkit is now available.

If you're not familiar, the Rapid XAML Toolkit is my side-project that's grown out of control. It provides a number of tools to help developers working with XAML files inside Visual Studio.

In addition to a load of bug fixes, this version adds more functionality for working with the XAML files of a Xamarin.Forms project.

The image below shows examples of some of the things that it can now identify inside the editor.

Screenshot showing issues highlighted in the editor

In the screenshot above you'll see that it highlights a number of different issues as either suggestions, warnings, or errors. Wherever possible it includes automatic fixes or will point you to more information about why the issue has been highlighted. Like Roslyn Analyzers in C#, you can also selectively disable or change the error levels of each issue too.


Join the more than 1500 people who use it every day - INSTALL it now!


Why v0.12? Why not v1.0 yet?

I'm trying to balance functionality and expectation.

It's not a preview but it doesn't yet have everything that I want it to. Once it includes examples of all the areas that will show its full potential, I'll upgrade it to version 1.0.