Detroit Dave

Reason #1,287,529 WSL Still Vexes Me

Tue, 21 Mar 2023 00:00:00 +0000

WSL is sometimes a very useful tool for working in a Linux environment. It is super useful because it is light weight and integrated in Windows. The drawback to using WSL is from time to time you run across some inconsistencies that are endlessly frustrating before you can “bottom out” the problem. Once figuring it out, it is of course evident where the problem came from but just makes you shake your head and remind you that doing anything cross-platform in WSL is “mostly” cross platform but not always. So, what did I discover and how did we fix it.

My task was to put together a bash shell script that used the Azure CLI to work with Azure Managed Applications. To create some of the variables needed for the bash script we set up some variables that were az cli commands getting a single piece of data. Now please remember, I am by no means a Bash expert and know simply enough to be dangerous or at the very least get myself in trouble. Let’s take a look at WSL and the commands used to set up the variables

You can see that that I ran the data collection part of the script on the first line. It simply returns a string that is the subscription id and puts in the variable called appid (line 1). Next, I build a second variable called managedGroupId using that first variable essentially inserting it into the string represented there (line 3). You can see the results of echoing the variable managedGroupId. What in the Sam Hill is that output!?!?!? I kept thinking “well maybe because the command result is async”, “maybe it is over writing the buffer on timing”, etc….. As I said, I am no Bash expert so I was perplexed thinking there was some Bash assumptions that I was overlooking. It took me roughly 4 hours to figure out the problem. The problem was derived from one Stackoverflow thread (here for proper attribution). The problem is that since you are using WSL it appears that the windows command’s output (even though this is WSL) has a trailing (\r, 0x0D, ^M) character that makes the line fold back and continue at pos 1.

The way to fix the Bash script to overcome the WSL problem (and not torch your script when you run on Linux) is to pipe it into another command to remove the trailing like so:

tr -d $'\\r'

This produces the desired output like so:

I hope this shows up on the internet and saves some time for everyone! 😊

Azure Devops Tidbits

Sun, 06 Sep 2020 00:00:00 +0000

I was recently working on a project that invovled doing a lot of work inside Azure DevOps (AzDO). We had some unique challenges to solve and I continually ran into trouble looking for ways to treat the AzDO Git repository as I would if I were going to download files from a Git repository (using the raw endpoints). Since I struggled finding a solution, and was only able to solve with a steer from a collegue, I thought I would publish an article so others could find.

AzDO Git and curl

This was the one that had me spinning for a while. If you take a look at the docs for the REST endpoints you would think you could grab the curl example and rock and roll. My experimenting proved that to be wrong. Take a look at the example below:

curl -u {username}[:{personalaccesstoken}] https://dev.azure.com/{organization}/_apis/projects?api-version=2.0

So, how do I use this? Are the ‘{}’ supposed to stay? Are they supposed to go? What about the ‘[]’? The only thing I found that worked for this was simply to use:

username:pat

Once this was done. I could get into most of the REST API end points.

Single File downloads

This one was a bit more interesting. The first thing you have to try to suss out is where is the endpoint to use. This was not as simple. With a steer from a collegue I found it here (replace all caps with your information):

https://dev.azure.com/ORGNAME/PROJECT/_apis/git/repositories/REPOSITORY/items/PATH-TO-FILE?versionType=Branch&versionOptions=None

Using the same curl -u format as above will get you any file you need form the repository. As an interesting side note, you could also do the curl command like so to the same effect. I have not tested this everywhere and have only used this format to get files (replace all caps with your information):

curl -o FILE -s https://PAT@dev.azure.com/ORGNAME/PROJECTNAME/_apis/git/repositories/REPOSITORY/items/PATH-TO-FILE?versionType=Branch&versionOptions=None

An interesting trick to execute code in a pipeline

We also found ourselves in a situation where we needed to do something in a build pipeline that was difficult to do in bash while running a build pipeline. This may be simply that I am not a bash God but we foind updating a json file in the build was difficult. Below is the dirty hack we used to get this done. :-)

We decided to use a little C# script (CSX) to solve our problem. The first thing we need to do was place a CSX file in the repository we were looking at. In this case the script was simply going to update some JSON:

#r "nuget: Newtonsoft.Json, 12.0.3"

using System;
using System.Collections.Generic;
using Newtonsoft.Json;

public class Extension
{
    public string name { get; set; }
    public string description { get; set; }
    public string type { get; set; }
    public string version { get; set; }
    public string filePath { get; set; }
}

public class Catalog
{
    public List<Extension> extensions { get; set; }
}

if (Args.Count != 3)
{    
    Console.WriteLine("Three arguments must be passed in: <index.json> <name> <version>");
    return -1;
}

Catalog catalog = JsonConvert.DeserializeObject<Catalog>(File.ReadAllText(Args[0]));
var extension = catalog.extensions.FirstOrDefault(x=> x.name.Equals(Args[1]));
extension.version = Args[2];
File.WriteAllText(Args[0], JsonConvert.SerializeObject(catalog, Formatting.Indented));
Console.WriteLine("Updated");

Once this was in the repository we needed to two more things: 1) get the tool to run the scripts and 2) run it. To install the NET Core SDK tool in your pipeline simply run:

_curl -s https://raw.githubusercontent.com/filipw/dotnet-script/master/install/install.sh

sudo bash_

From there you can simply execute the above script (in this case called ) like so:

sudo dotnet script ./update-version.csx extensions/index.json “core” $(Build.BuildNumber)

Wrap Up

I had a heck of a time trying to track down how to do this. I hope any of this is helpful to someone or at least finding it is easier.

A simple example of Azure B2C integration with Flutter

Sun, 19 Apr 2020 00:00:00 +0000

The intro

I love Azure. I mean, sometimes it is a love/hate thing, but I really LOVE Azure. :smile:

Don’t get me wrong, I like other cloud platforms as well, but I find it, like most things Microsoft tends to do in the developer space, easy for devs to get into. After years of doing my own authentication systems and integration of others, I was thrilled when Azure Active Directory B2C came out. It was especially great for putting that same user identity and security into many mobile apps I have built. As I have spent time adopting Flutter, it seems the documentation out there on how to integrate and use it more fully is spartan at best and scattered and erratic at worst. I currently have someone who asked to be an intern with me and learn the “craft” of coding, and I found getting them up to speed on this was time consuming. I thought I would document what it takes to stand up a B2C directory and integrate it into a Flutter app from an extremely simplistic perspective. In subsequent articles, I will move up the chain in complexity. You might wonder when you read this, “why is he taking me into one document and then jumping out and to another?”. The reason for this is actually simple. Azure B2C is a wide and varied product. It has a TON of different functionalities. I found it difficult to seperate the “wheat from the shaff” when reading through all these. It was always the “extra” information that threw me off, as I could not figure out what one thing had to do with another. I hope you enjoy the journey, and I hope you find this useful.

The premise

So what do we want to accomplish? The business desire here is to:

create a mobile app in Flutter where our users log in via an Azure B2C directory we create
access users’ profile information from the mobile application
allow users to log out of the app

This in a nutshell is the simplest use case for a mobile app. you might ask youself, “well, what about calling back end APIs and such?”. I wanted to start with the simplest thing— in my experience, trying to wind all of that in at one time was confusing and made troubleshooting harder while learning.

What technology will we be using?

We will be using:

Azure Active Directory B2C - https://azure.microsoft.com/en-us/services/active-directory-b2c/
Flutter - https://flutter.dev/
App Auth - https://appauth.io/

Getting B2C up and running

Create B2C Directory

We start by creating the Azure Active Directory B2C instance. Microsoft has a ton of documentation. I would start here to get the base directory in place. I have named mine detroitdaveb2c.

Register the Mobile Application

The next thing we have to do is create/register an application and create the user flows within this directory. The process to create a web app and create a native app are very different. This was one of the confusing things to get through as there are a ton of options. This article is a good one to follow to get those steps correct. While creating/registering the application, these tips may be useful:

Pay attention to the redirect URI in your app. You will find different articles with different ways of laying this out. One of the examples used is com.onmicrosoft.contosob2c.exampleapp://oauth/redirect. The first part before the :// is not that important, and you can make it what you wish (though unqiue for a mobile device namespace). I would recommend you leave the last part as it is. I tried doing things like using: ://oauth or ://redirect as other articles have stated, and they cause various technical problems with libraries used.
DO NOT download their sample code. I found it confusing to match up the code on the Flutter side. I suppose if you are an expert Objective-C or Android guru and have some experience with AppAuth it might help. As I did not, I experienced nothing but pain trying to make it run and work.

Your application should look something like this:

I would follow this article as the way to make this user flow. For purposes of our tutorial let’s name it “susi” and pick the following attributes:

Collect attribute:
- Display Name
- Given Name
- Surname
- Email Address
Return claim:
- Display Name
- Given Name
- Surname
- Email Address
- User’s Object ID
- User Is New

Your newly created user flow should look something like this:

Flutter App

Create the app

You can use either VS Code or Android Studio to create the Flutter application. Use either one to create the basic application and we will work from there. In VS Code you can use the flutter CLI to create your project. Screenshots of the Android Studio way are below:

This is the base Flutter app where we press the button that keeps a variable with how many times we have clicked it. We will use this as the starting point for our app.

Integrate App Auth into our App

To begin modifying this simple application to suit our needs, we will learn how to use AppAuth inside our Flutter application. To make this work, we will use the flutter_appauth library. First thing to do is update your pubspec.yaml file with the latest package information. If you go to the library’s GitHub site you can find directions on how to integrate it. I will also be walking through the integration process. AppAuth is a fairly complicated library with lot of knobs and dials. The first step is to add the import of the AppAuth library and add some variables to hold all our information. We can place this at the top of the _MyHomePageState class replacing the increment variable:

  FlutterAppAuth appAuth = FlutterAppAuth();
  String _clientId = '90185a76-af9b-4ff0-9fdc-55c888583e2d';
  String _redirectUrl = 'com.davetest.wow://oauth/redirect';
  String _discoveryURL = 'https://detroitdaveb2c.b2clogin.com/detroitdaveb2c.onmicrosoft.com/v2.0/.well-known/openid-configuration?p=B2C_1_susi';
  String _authorizeUrl = 'https://detroitdaveb2c.b2clogin.com/te/detroitdaveb2c.onmicrosoft.com/b2c_1_susi/oauth2/v2.0/authorize';
  String _tokenUrl = 'https://detroitdaveb2c.b2clogin.com/te/detroitdaveb2c.onmicrosoft.com/b2c_1_susi/oauth2/v2.0/token';
  String _idToken;
  String _codeVerifier;
  String _authorizationCode;
  String _refreshToken;
  String _accessToken;
  String _accessTokenExpiration;
  String _firstName = "";
  String _lastName = "";
  String _displayName = "";
  String _email = "";
  Map<String, dynamic> _jwt;
  List<String> _scopes = ['openid'];

For _clientId I use the application ID for the application created in the Azure B2C portal. For _scopes please note that in Azure B2C, a base directory instance only has the ‘openid’ scope. The discovery and endpoint URLs are worth a longer discussion. In any OAuth server you have the concept of autodiscovery. Where might I find the discovery endpoint for Azure B2C? If you go to the user flow page you can see it.

Go take a look at yours. The important bits of mine are below

  "authorization_endpoint": "https://detroitdaveb2c.b2clogin.com/detroitdaveb2c.onmicrosoft.com/oauth2/v2.0/authorize?p=b2c_1_susi",
  "token_endpoint": "https://detroitdaveb2c.b2clogin.com/detroitdaveb2c.onmicrosoft.com/oauth2/v2.0/token?p=b2c_1_susi",
  "end_session_endpoint": "https://detroitdaveb2c.b2clogin.com/detroitdaveb2c.onmicrosoft.com/oauth2/v2.0/logout?p=b2c_1_susi",

The problem is if you try to use any of these endpoints, your program will work in vary degress of success across both platforms. There is a hidden gem in the article we used to create the application in B2C. This hidden gem says:

The token endpoint URI can be generated by replacing the Tenant_ID and the Policy_Name in the following URL:

static NSString *const tokenEndpoint = @"https://<Tenant_name>.b2clogin.com/te/<Tenant_ID>/<Policy_Name>/oauth2/v2.0/token";* 

This format of the endpoint is different than what you see from the portal and this format was the only thing I was able to get to work across both iOS and Android reliably.

Next let’s change the _incrementCounter() method to the follow code:

  Future<void> _logIn() async {
    try {
      final AuthorizationTokenResponse result = await appAuth
          .authorizeAndExchangeCode(
        AuthorizationTokenRequest(
          _clientId,
          _redirectUrl,
          serviceConfiguration: AuthorizationServiceConfiguration(_authorizeUrl, _tokenUrl),
          scopes: _scopes,
        ),
      );
      if (result != null) {
        _processAuthTokenResponse(result);
      }
    } catch (e) {
      print(e);
    }
  }

IMPORTANT NOTE: Obviously a lot of the examples use the auto discovery URL and parameters to make things go. The ONLY way I was able to get this to work reliably with Azure B2C was to use the endpoints talked about above.

In our program we can now add a few methods to decode what comes back:

  void _processAuthTokenResponse(AuthorizationTokenResponse response) {
    setState(() {
      _accessToken = response.accessToken;
      _refreshToken = response.refreshToken;
      _accessTokenExpiration = response.accessTokenExpirationDateTime?.toIso8601String();
      _idToken = response.idToken;
      //get individual claims from jwt token
      _jwt = parseJwt(response.idToken);
      _firstName = _jwt['given_name'].toString();
      _lastName = _jwt['family_name'].toString();
      _displayName = _jwt['name'].toString();
      _email = _jwt['emails'][0];
    });
  }

  Map<String, dynamic> parseJwt(String token) {
    final parts = token.split('.');
    if (parts.length != 3) {
      throw Exception('invalid token');
    }
    final payload = _decodeBase64(parts[1]);
    final payloadMap = json.decode(payload);
    if (payloadMap is! Map<String, dynamic>) {
      throw Exception('invalid payload');
    }
    return payloadMap;
  }

  String _decodeBase64(String str) {
    String output = str.replaceAll('-', '+').replaceAll('_', '/');
    switch (output.length % 4) {
      case 0:
        break;
      case 2:
        output += '==';
        break;
      case 3:
        output += '=';
        break;
      default:
        throw Exception('Illegal base64url string!"');
    }
    return utf8.decode(base64Url.decode(output));
  }

Let’s add the code to log out of B2C. Add a method to log out.

  Future<void> _logOut() async {
    try {
      //for some reason the API works differently on iOS and Android
      Map<String,String> additionalParameters;
      if (Platform.isAndroid) {
        //works on Android but is missing p parameter when redirected back to authorize on iOS
        additionalParameters = {
          "id_token_hint": _idToken,
          "post_logout_redirect_uri": _redirectUrl
        };
      } else if (Platform.isIOS) {
        //missing p parameter when redirected back to authorize on iOS so the below difference
        additionalParameters = {
          "id_token_hint": _idToken,
          "post_logout_redirect_uri": _redirectUrl,
          'p':'B2C_1_susi'
        };
      }
      await appAuth.authorizeAndExchangeCode(
        AuthorizationTokenRequest(
          _clientId,
          _redirectUrl,
          promptValues: ['login'],
          discoveryUrl: _discoveryURL,
          additionalParameters: additionalParameters,
          scopes: _scopes,
        ),
      );
    } catch (e) {
      print(e);
    }
    setState(() {
      _jwt = null;
    });
  }

A couple of words here. OAuth2 makes use of the browser for security reasons. It is a long story and a fun read if you are bored. The logout through the AppAuth library is not seemless. It logs you out, but the way to make the single API work is to redirect back to the home page so the web “page” can return something to the UI. This makes the user press the logout button, be redirected to the login page, then press the “X” or “Cancel” to get out of it and back to the app. In my humble opinion, this is a bad user experience for everyone. I have read a lot of “issue” threads on GitHub for AppAuth on the things people have done to get around it, namely forking the code and adding their own stuff. I deemed that out of scope here and wanted to keep things simple even if the log out is a little “messy”.

Once again, the only thing that really works in this case is the discoveryURL parameter. This is becasue if you look at the underlying Flutter code in the flutter_appauth library, the AuthorizeTokenRequest method takes an AuthorizationServiceConfiguration that is both an auth endpoint and token endpoint. You can pass it a logout endpoint, but it doesnt work well across both platforms. Lastly, we have to do a little ‘dirty hack’ coding here. The problem is on Android I don’t have to pass the “p” parameter in to get it to work, but then it would not work on iOS. More frustratingly, if I did the opposite it would work on iOS but not Android. To fix this, I did a little “platform checking” (which in my honest opinion should be avoided when writing Flutter) and change the parameters being passed. The good news is this works like a charm.

Moving forward, let’s add a bit of code to display what was was returned as well as our state. In the build() method within the _MyH, let’s replace the children property where the increment counter is with the following code:

    children: (_jwt == null) ?  <Widget>[ Text('Please press + sign to log in',)] :
    <Widget>[
    Text(
        'Display Name: $_displayName',
    ),
    Text(' '),
    Text(
        'Name: $_firstName $_lastName',
    ),
    Text(' '),
    Text(
        'Email: $_email',
    ),
    Text(' '),
    RaisedButton(
        onPressed: _logOut,
        elevation: 5,
        child: Text('Logout'),
    )
    ],

And lastly in our list, to make our app run on Android we need to update update our Android’s build.gradle file and update our info.plist on the iOS side. Your shceme in both of these cases is everything you put before the :// in the redirect URI.

This ends up looking like:

I hope you have enjoyed this post. I will expand on this with other libraries and other B2C complexities in future posts. The code for this article can be found here. Comment below if you have questions or concerns. Thanks for reading.

Trying Something New

Sun, 29 Mar 2020 00:00:00 +0000

Today is my birthday. I turn 50 years old today (damn!!). I spend a lot of time pouring out what I know to customers, partners, and younger programmers. My current manager encourages me to share more of what I know. Today, I have decided to start to write down some of what I do and what I know from both my time at Microsoft and things that happen outside of it.

I hope it is mildly useful to anyone and I hope you like it. Maybe all it will be is simply a way for me put things down on paper so I don’t forget as I get to older. Maybe…….. where’s my pudding cup????