The Musings of Marc

Setting Content-Type for multipart/form-data values

Marc Savy — Thu, 28 Mar 2019 00:00:00 GMT

I was recently creating a form that submitted two elements at once: a JSON payload, and a file. My back-end kept choking on the JSON payload, claiming it did not know how to decode it. That seemed odd, to say the least.

My first attempt looked something like this:

    const file ....
    const jsonPayload: String = JSON.stringify(objToSend);
    const formData: FormData = new FormData();

    formData.append('jsonPayload', jsonPayload);
    formData.append('my-file', file);

    return this.http.post(`${this.resourceUrl}`, formData, {
        observe: 'response'
    });

This appears to work, however your backend will likely start complaining about the jsonPayload field. Why?

Let’s see what the browser sent:

------WebKitFormBoundaryJU3pJb6zPtyXf5GH
Content-Disposition: form-data; name="jsonPayload"

{"foo":"bar"}

------WebKitFormBoundaryJU3pJb6zPtyXf5GH
Content-Disposition: form-data; name="my-file"; filename="something.csv"
Content-Type: text/csv (1)

------WebKitFormBoundaryJU3pJb6zPtyXf5GH--

At first blush, everything seems okay; the data is there, and the names seem correct. However, on closer inspection, notice that my-file has a Content-Type set at (1), whilst jsonPayload does not.

By default, most browsers will infer a File Content-Type correctly, but will not set a the Content-Type for the jsonPayload string.

Unfortunately, the standard append String method on FormData doesn’t seem to let us set the content type; so we must take a slightly more obscure route by creating a new Blob, into which goes our jsonPayload — setting the content type in the subsequent options parameter:

   formData.append('jsonPayload', new Blob([
       jsonPayload
   ], {
       type: "application/json"
   }));

Now it should be behaving as expected:

------WebKitFormBoundaryJU3pJb6zPtyXf5GH
Content-Disposition: form-data; name="jsonPayload"; filename="blob"
Content-Type: application/json (1)

{"foo":"bar"}
------WebKitFormBoundaryJU3pJb6zPtyXf5GH
Content-Disposition: form-data; name="my-file"; filename="something.csv"
Content-Type: text/csv


------WebKitFormBoundaryJU3pJb6zPtyXf5GH--

1	Now our JSON payload has the correct Content-Type, hooray!

Farewell, Red Hat friends!

Marc Savy — Fri, 18 Jan 2019 00:00:00 GMT

This is an approximate reproduction of the email I sent to my soon-to-be erstwhile Red Hat colleagues:

This is a difficult email to write. I’ve had the great fortune to be part of Red Hat for over 8 years, and whilst there have been many challenges and accomplishments along the way, the one constant has been the wonderful people.

For many, colleagues are only ever acquaintances; it’s a tribute to you all that many have become personal friends. I am certain we’ll stay that way.

Whilst it’s time for me to move on and embrace a new challenge, I very much hope this isn’t the last I will see of you all. Whether in person, or in the open source community.

I have so many of you to thank for helping me develop my career, and more importantly, being a significant presence outside of work; always offering good advice, encouragement, and advocating for me. Truly, thank you.

I’d like to make special mention of Eric, who has been a truly excellent person to collaborate with, and has helped me develop in ways I wouldn’t have imagined. You could not wish to work with a better person. We nearly got over the line, but not quite!

There are so many other Red Hatters who have had very a positive influence on my career and life, and it’s difficult to choose who to mention, but I’d like to explicitly call out a few: Julien and all the Vert.x team, Sanne, Stian and the KC team, Andrew Dinn, Dave, MarkL, Keith Babo, Keith Lynch, Kev, Ken, Bob, Lance, Ben & Co, Rachel, all my friends at Newcastle, our excellent friends in QE, and all those that advocated for our work. There are so many others, and your name not explicitly being here isn’t a sign of anything other than my attempt at brevity.

If you have time, I would love for you all to check out Apicurio (https://www.apicur.io/ — click "try live"); it’s Eric’s latest project, and I’ve been assisting the last month or so. It’s a rich UI, editor, and validator for creating and collaborating on OpenAPI (fka Swagger) documents; moreover, it can generate code and is increasingly providing slick integration with other Red Hat projects. It’s really at the head of the pipe when it comes to the developer experience, and I think it’s a fantastic addition to the Middleware portfolio.

Finally, I wish you all the very best of luck with your endeavours. Your technical talents, understanding of open source and community, and continued hard work will be amongst the key attributes that I hope will allow Middleware to prevail.

Regards,

Marc

I was involved in a shocking incident and want to talk about mental health - and what we're doing as a country to address it.

Marc Savy — Fri, 18 May 2018 00:00:00 GMT

This post covers sensitive topics including attempted suicide and mental illness.

Please, if you ever have suicidal thoughts, reach out to an organisation such as The Samaritans. People truly care about you.

On 7th May, at about 10pm, I pulled back a suicidal man who was jumping off the Redheugh Bridge in Newcastle/Gateshead.

It was one of the most extraordinary and frightening moments in my life.

I wasn’t intending to go out at all on that day. At the last minute a friend called me and we decided to meet up. One thing led to another and we ended up going for a walk down the river and grabbing a few drinks. We then decided to go for something to eat, but the first choice was closed; we ended up going to a Japanese restaurant. The food was slow as it was very busy, so it was much later than expected when I started heading back home.

There’s a narrow walkway along the bridge, with 4 lanes of traffic and chest-high barriers to either side. I saw a large, shirtless, and shoeless man walking the opposite way to me. He suddenly straddled the barrier.

I ran towards him, grabbed his arm and started talking to him; trying to convince him not to jump. I said I wanted to hear what he had to say, and that I could see that he is in a dark place, but his life is valuable and important, and that although things seem awful now, they can improve.

He was talking back to me, but refused to come down; saying he was going to jump and kill himself.

A lady who seemed to have encountered him before ran up and also tried to persuade him to come down.

We were there for a good while talking back and forth as I tried to convince him, but he wasn’t coming back.

He eventually started objecting to me holding his arm, and kept pushing me off. So, I moved slightly behind him and tried to position myself as best I could.

Suddenly he went for it and started dropping off the side.

I had to try something, as things seemed to have reached a point of no return. I reached over and yanked him with all my force and, fortunately, dragged him back over, off the barrier, and onto the floor of the walkway.

He was pretty angry for a few moments, and he was a lot larger than me, so I probably took my chances. After a short while he started crying and said he is bipolar. I managed to keep him on the floor. He clearly was in a terribly dark place and mentioned awful reasons contributing to his state that I won’t repeat here.

I’m so glad for his sakes, and mine, that he didn’t go off and I was able to stop him. I think I would have been messed up for life if I’d seen him fall and die.

A passerby came as I was dragging the guy back over, and he called the emergency services. My phone had died so, unfortunately, I didn’t get to collect his details. I’m also going to reach out to the officers to see if I can send him my contact information to thank him in person and talk about what happened.

The Police arrived for the ill man soon after and took some of my information.

The suicidal man shook my hand and thanked me as they took him to hospital. One of the officers was a bit impatient with him because he wasn’t being completely cooperative. I guess they get compassion fatigue from seeing this sort of thing so frequently. I imagine the most challenging types of compassion and empathy are with types of people we aren’t innately sympathetic towards, and who don’t necessarily cooperate.

The officers gave me a lift back home and suggested I "have a stiff drink". I was so amped up I couldn’t sleep for several hours; felt weird for a few days, but I’m perfectly okay now.

It really strikes me that my coincidental encounter with this man was promulgated by a huge sequence of unusual events and timings. With the smallest change I would have missed him, or worse, seen him drop. I wasn’t planning to go out at all, let alone planning to be out late.

I’m writing about this for a few reasons:

Firstly, we need to do much better at looking after people with severe mental health issues. I’ve become increasingly concerned over the last few years about underfunding of mental health care causing people who ought to be receiving intensive levels of care to be sent home when it is not in their best interests. I hope by recounting this it makes some people rethink how much they are willing to contribute to ensure we are able to offer sufficient care to vulnerable people. I truly hope he gets the care he needs - given the extremity of the issue hopefully he’ll be prioritised.

He said he’d felt suicidal many times before but nobody had taken him seriously. I’d guess his perception is likely shaped through the lens of his illness, but the fact he was wandering around half-naked in such a distressed state is a terrible shame.

I hope everyone would try to help a stranger in such a state. I asked a couple of (homeless, I think) guys walking past to help, but they just ignored me and walked off. We’re nothing if we don’t help one another.

People with addiction and mental health issues are often at the bottom of the pile in society. This encounter really reminded me that we need to be better to one another and do more to address MH concerns. I’m cognisant that many people are extremely difficult to deal with and there are no easy solutions, but perhaps we don’t have the balance right.

I think the barriers are far too low on the Redheugh bridge. I realise that a thoroughly determined person will defeat any measures, but my understanding is that many suicides are spur-of-the-moment, and modest impediments can help delay their actions long enough for them to find the clarity they need to reconsider. Perhaps they need to think about nets or similar devices that have been used successfully in other locations. I plan to bring it up with my MP, but I’m sure it ultimately comes down to a lack of funding and/or structural issues.

Follow-up

I went down to the police station the next day to give them my phone number. An officer emailed later asking what I wanted to know, but then didn’t bother responding to my message. I had wanted to make myself available to give a full statement and record the very serious things the man had told me; they would have been relevant to his treatment team (and likely legally). I guess it was too much hassle.

If you made it this far: thank you for reading.

Introducing asciidoc-admonition-icons, a GitBook plugin to restore font icons for AsciiDoc admonition blocks.

Marc Savy — Sun, 07 May 2017 00:00:00 GMT

I’ve been working hard on the apiman 1.3.x documentation. One nice feature of AsciiDoc/Asciidoctor are admonition blocks; allowing visually clear and categorised emphasis for the important points.

However, at the time of writing this blog, the GitBook AsciiDoc 3.x.x rendering pipeline does not support icons for admonition blocks, providing an inconspicuous, text-only representation:

There are some PRs and GitHub issues with people trying to get this fixed, but none of them has been merged, so I really wanted to provide an interim solution that would still work on the gitbook.com hosted platform.

Plugin to the rescue.

The solution was to create a simple plugin, asciidoc-admonition-icons ^[1], that identifies admonition blocks and rewrites the label element to contain the missing glyphs:

The default styling provides a similar look-and-feel to the Asciidoctor’s standard theme; with FontAwesome icons and a comparable colour scheme. Naturally, you can override both the glyphs and styling.

This should hopefully provide a satisfactory workaround until the GitBook team resolve the issue properly in their codebase.

If you have any problems, please feel free to file an issue.

Installation & Configuration

In your book.json, put asciidoc-admonition-icons in your plugins array

{
	"plugins": ["asciidoc-admonition-icons"],
}

Provide your overrides in pluginsConfig.admonitions:

The 5 standard AsciiDoc admonition types are available:
classes: CSS classes of the admonition icon container.
title: HTML title attribute of admonition icon container.
content: Unicode value of glyph to use.

The entire set of defaults are shown here for illustrative purposes, but you need only provide those you’re overriding:

Example book.json configuration:

"pluginsConfig": {
    "admonitions": {
        "note": {
            "classes": "fa icon-note",
            "title": "Note",
            "content": "\uf05a"
        },
        "tip": {
            "classes": "fa icon-tip",
            "title": "Tip",
            "content": "\uf0eb"
        },
        "important": {
            "classes": "fa icon-important",
            "title": "Important",
            "content": "\uf06a"
        },
        "caution": {
            "classes": "fa icon-caution",
            "title": "Caution",
            "content": "\uf06d"
        },
        "warning": {
            "classes": "fa icon-warning",
            "title": "Warning",
            "content": "\uf071"
        }
    }
}

For example, if we were to override the important icon with another glyph:

"important": {
	"classes": "fa icon-important",
    "title": "La bañera es muy importante!",
    "content": "\uf2cd"
}

Check out the plugin’s GitBook plugin page, GitHub repo or NPM readme for detailed instructions.

Despite being simple code, quite interesting customisations are possible using a combination of configuration options and CSS. I might add some more sophisticated examples in a future blog post (refer to the docs for some simple ones).

1. Full NPM package name is: gitbook-plugin-asciidoc-admonition-icons

Generic JWT policy for apiman

Marc Savy — Thu, 01 Dec 2016 00:00:00 GMT

I just pushed an initial (very simple) generic JWT authentication plugin policy to master. It should work with any JWT provider! Let me know how it works for you.

Update on 23/01/07: The JWT plugin has now been released to Maven, you can install it via:

G: io.apiman.plugins
A: apiman-plugins-jwt-policy
V: 1.2.9.Final (or whatever the latest/matching version of apiman is)

Old Instructions:

Build and install

To try it out immediately you will need to build it from source. Just check out the apiman/apiman-plugins repo and execute:

mvn clean install

The plugin coordinates will be:

G: io.apiman.plugins
A: apiman-plugins-jwt-policy
V: 1.2.9-SNAPSHOT

It isn’t yet as feature-rich as the Keycloak plugin, but you can:

Require JWT.
Require claims (e.g. sub = foo).
Require transport security (TLS, SSL).
Require JWT be cryptographically signed (aka. JWS).
Validate JWT against a provided public key.
Remove auth tokens (prevent them reaching the backend).
Set maximum clock skew.

I’ll expand on this in the near future to add some commonly-used features from the Keycloak plugin:

Extraction of roles for authorization
~~Forward token fields as headers (e.g. X-Sub = sub)~~ ✓

How to use it

Either:

Set the Authorization header as Authorization: Bearer
Set the access_token query parameter as example.com?access_token=

Apiman's New Execute Blocking Component

Marc Savy — Sun, 27 Nov 2016 00:00:00 GMT

The Execute Blocking component is a small but potentially useful new component I’ve written and committed to the apiman gateway, allowing custom policy plugin authors to safely execute blocking code.

Blocking is bad, but…

Apiman’s gateway embraces non-blocking and asychronous design principles in order to best exploit the beneficial performance characteristics of platforms such as Vert.x.

A constraint of this approach means that code must not block the thread it is executing upon for excessive periods of time, lest an entire reactor (and often physical core) becomes completely unavailable; blocking on IO or holding locks for example. For those who are interested I explored this issue in more depth in a previous post, but safe to say, this golden rule can be difficult to abide by in the real world where libraries and protocols outside of our control may not behave as we need.

The Execute Blocking Component (IExecuteBlockingComponent) comes to our rescue, providing a mechanism to execute blocking code from a threadpool rather than directly on the reactor.

Needless to say, special attention should be paid to its limitations and characteristics. Only use this functionality when strictly necessary, and with the smallest practical scope, as it substantially reduces the performance benefits of the underlying non-blocking architecture.

Execute Blocking

Here’s a short example of the component in action:

void doApply(ApiRequest request, IPolicyContext context,
    IPolicyChain chain) {

    IExecuteBlockingComponent ebComponent =
        context.getComponent("IExecuteBlockingComponent.class");

        ebComponent.executeBlocking(future -> { (1)
            String result = BlockingService(50000); (2)
            future.completed(result); (3)
        },
        result -> { (4)
            if (result.successful()) {
                request.getHeaders().put("Foo", request.getResult()) (5)
                chain.doApply(request);
            } else {
                chain.doFailure(new PolFailure(result.getError())); (6)
            }
        });
}

1	Execute your blocking code within the first lambda, and indicate to the `future` when your code has terminated, either `completed` or `fail`. Exceptions thrown within the scope of the lambda will be caught and implicitly passed to `fail`.
2	Imaginary blocking service that blocks for 50000ms.
3	Indicating the future has completed successfully and passing the result to `completed`.
4	The result is passed to this second lambda asynchronously.
5	Set our successful result as a header `Foo`.
6	Creating a policy failure with the error.

Only use executeBlocking when absolutely necessary and constrain it as tightly as possible.

Apiman on blocking platforms

You may be wondering what happens if you use executeBlocking on an apiman platform which has backed by a implementation where it’s safe to block the executing thread (such a servlet on WildFly). Simply, a passthrough implementation is used on those platforms which executes immediately on the same thread - so it’s safe to use on any platform.

Even if you tend to run your plugins on a blocking platform you should still respect the "no blocking the main thread" rule and use executeBlocking, as other platforms and users will expect this guarantee - it’s especially crucial if you wish to share your work with the community.

Final Words

This is the first commit of this code onto master, so changes are possible. This feature is directly designed to enable the use of Vert.x, so their interface has been adopted fairly closely, so many thanks to them again for their brilliant work which we’re fortunate to be able harness in apiman.

Stopping Vert.x BlockedThreadChecker exceptions during interactive debugging

Marc Savy — Mon, 24 Oct 2016 00:00:00 GMT

Vert.x’s standard verticles require non-blocking code to work optimally. A watchdog will warn you (extremely insistently) if anything does block for an excessive period. However, if you’re just trying to run an interactive debugger ^[1] then this can become extremely annoying as your console is flooded with io.vertx.core.impl.BlockedThreadChecker exceptions that look something like this:

Sections

What’s the cause?
Workarounds
- Via System Properties
- Directly in VertxOptions
Caveats & Conclusion

What’s the cause?

The exceptions occur because the interactive debugger repeatedly halts the event loop thread while you use it: browsing around, poking at variables, stepping through your program, etc. The length of time it takes to execute the event being debugged will be drastically longer than normal; hence the watchdog starts barking.

Mercifully there’s a simple solution, which involves setting blockedThreadCheckInterval to a sufficiently long time. ^[2]

The following should only be used for debugging purposes!

If your production code is throwing BlockedThreadChecker exceptions, then you have a problem you need to fix.

Workarounds

Via System Properties

Whether you’re using Launcher (which has supplanted Starter) or the Vert.x CLI, pass the following VM argument:

-Dvertx.options.blockedThreadCheckInterval=200000000 (1)

1	Some sufficiently large integer (milliseconds).

All IDEs also support setting system properties in some fashion. I’ve briefly outlined Eclipse and IntelliJ below, but the procedure should be similar for others.

Eclipse Debug Configurations → Arguments → VM Arguments

IntelliJ Edit Configurations → Select Application → Configuration → VM Options

Setting blockedThreadCheckInterval via system properties is likely the best approach; avoiding debug-specific behaviour leaking into your code.

Directly in VertxOptions

If you are not using the CLI or Launcher, or for some reason wish to avoid using system properties, then you can simply set the interval in VertxOptions directly.

VertxOptions vxOptions = new VertxOptions()
	.setblockedThreadCheckInterval(200000000); (1)
Vertx myVertx = Vertx.vertx(vxOptions); (2)

1	Set interval in `VertxOptions`.
2	Pass options to `Vertx.vertx()` factory.

Caveats & Conclusion

It’s worth remembering that by halting executing thread(s) in the debugger you may not encounter timing-related issues you otherwise would. So, it is wise to ensure you aren’t altering blockedThreadCheckInterval unless you really need to.

Needless to say, seeing BlockedThreadChecker warnings in production is a symptom of pathological code behaviour, indicating that the event loop is being blocked for excessive periods; you should fix it rather than altering timeouts.

Finally, you should normally prefer the system properties approach so that you can isolate this change to your debugging profiles only.

1. Eclipse, IntelliJ, NetBeans, etc.

2. Had this as maxEventLoopExecuteTime in a previous revision of the blog, but this doesn’t work consistently.

Vert.x and Blocking Code

Marc Savy — Tue, 18 Oct 2016 00:00:00 GMT

One of Vert.x’s core tenets is that you should never block the event loop. If you’re writing an application from scratch and limit yourself to Vert.x’s provided facilities you’ll not go far wrong.

However, if you’re writing a more complex application (such as apiman) there’s a good chance that you’ll be forced to stray outside of these boundaries, for example: integration constraints; inherently blocking protocols; static or shared data-structures. ^[1]

Luckily, the Vert.x team understand this, and have provided ways to work with these real-world requirements.

Back in the Vert.x 1.x and early 2.x days you only had one option: worker verticles. Now the team have expanded things and there are several excellent options; so, let’s have a quick review of them. ^[2]

Sections

Why is blocking bad, anyway?
Solutions

Why is blocking bad, anyway?

Vert.x uses a pragmatic implementation of the reactor model (more specifically multi-reactor).

A simple way to visualise this is to imagine there is only one thread per core, looping indefinitely, and executing work (handlers) from a queue. Many verticle instances share a given reactor; their execution is also enqueued as a task and runs asynchronously - that is, it only runs when the reactor has churned through any preceding tasks and reaches the verticle’s task(s). Each task runs on the reactor thread directly.

Hence, if any of those enqueued tasks executed a blocking operation, such as listening on a socket, the entire reactor would be unable to make progress until the thread resumed.^[3]

Instead, Vert.x deals with blocking operations outside of the event loop; enqueuing a handler to be invoked when the result of the blocking operation is completed ^[4]. For instance, instead of blocking on a socket until something arrives, we ask the operating system to call us back when something relevant arrives and invoke our handler with the result.

The key benefit of this approach is that the reactor thread does not need to block, so it can achieve excellent CPU utilisation - spending as much of its time as possible usefully executing, rather than waiting or being blocked.

In contrast, the typical webserver’s threads will spend much of their time in a non-running state (e.g. blocked or waiting), leaving large amounts of CPU time unused ^[5]. To combat resource under-utilisation developers harness large numbers of threads so that new requests can continue to be satisfied. However, this incurs substantial CPU, memory and latency costs due to frequent context switching ^[6].

But, what if you need to do a blocking operation and it’s not one of Vert.x’s supported things?

Solutions

Worker Verticles

The first approach Vert.x offered was worker verticles. Each worker verticle instance executes entirely on its own thread taken from a threadpool, hence when an operation blocks it does not affect the event loop or block progress.

If your verticle’s work is predominantly blocking, then a worker verticle is a good option. You can use the pattern outlined in the example below to send and receive work over the event bus - this maintains asynchronicity and non-blocking behaviour in your standard verticles whilst allowing you to offload long-running, blocking functionality onto worker verticles. You should aim to minimise the amount of code run in worker verticles to maximise overall resource utilisation. ^[7]

Offering a blocking service

package com.example.foo;

public class BlockingVerticle extends AbstractVerticle {
    @Override
    public void start() throws Exception {
        vertx.eventBus().consumer("com.rhymewithgravy.doesit",
            message -> { (1)
                boolean answer = SlowBlockingRhyme
                    .doesItRhymeWithGravy(message.body()); (2)
                message.reply(answer);
            } );
    }

    public static void main(String... args) {
        DeploymentOptions opts = new DeploymentOptions()
            .setWorker(true); (3)
        Vertx.vertx()
            .deployVerticle("com.example.foo.BlockingVerticle", opts); (4)
    }
}

1	Listening (non-blocking) on the event bus.
2	A service that blocks while it figures out if a word rhymes with gravy. Mostly the answer is no.
3	Setting the verticle to be deployed as a worker with `setWorker(true)`.
4	Deploying the verticle, as normal.

Consuming a blocking service

Here’s an example of the some non-blocking sender code which may interact with our fictional service over the event bus:

vertx.eventBus().send("com.rhymewithgravy.com.doesit", (1)
    "apples", (2)
    reply -> {
        System.out.println("Does apples rhyme with gravy? "
            + reply.result().body()); (3)
    }
);

1	Send a message to the service over the event bus.
2	Contents of message to send (i.e. word we want to send to rhyme service).
3	Print out the blocking service’s response: "no", I would presume!

Execute Blocking Statements

If the majority of your verticle’s work does not involve blocking (which is probably the most common scenario), then executing the entire verticle on a worker thread is rather wasteful. Even with clever optimisations it isn’t fully exploiting the platform’s strengths.

It is possible to work around this limitation by separating blocking and non-blocking code into distinct verticles, with the event bus acting as the asynchronous bridge to avoid the blocking verticle stopping progress in the non-blocking verticle ^[8]. However, this can be rather unwieldy and forces additional boilerplate and complexity where it isn’t desirable.

So, a newer and much more elegant solution was introduced for exactly these scenarios: executeBlocking.

Code in an executeBlocking handler will be run from a worker threadpool where it is safe to block, with the results passed back asynchronously.

vertx.executeBlocking(future -> {
    boolean result = FastBlockingRhyme.doesItRhymeWithGravy("apples"); (1)
    future.complete(result); (2)
}, res -> {
    if (res.succeeded()) {
        System.out.println("Does apples rhyme with gravy?  "
          + res.result()); (3)
    }
});

1	New improved rhyme engine, it blocks but is fast.
2	Indicate that the work in the block has been completed: successfully or otherwise.
3	The result pops out in the second lambda, and we print out a helpful message.

In this sample executeBlocking accomplishes the same work as the event bus pattern shown in the Worker Verticles example, but in a much more concise, efficient and less error-prone manner.

executeBlocking should be the preferred way of running blocking code for most simple use cases.

Vert.x Sync

Sync the newest area of development for handling blocking code in Vert.x, taking an extremely different approach to the aforementioned techniques. Quasar is used to offload blocking work onto fibres, which are extremely lightweight non-kernel threads that don’t incur the same context switching penalty as traditional threads.

It’s not an option which I’ve had the opportunity to explore in much depth yet, but I’m going to spend some time analysing it and follow up; for example, to understand performance implications, maintainability changes and deployment considerations. It sounds a promising area of research and development; asynchronous development tends to be trickier and can suffer from "handler inception" (excessive forwarding and handler hell) which Sync helps flatten out.

One of the most obvious differences is that you must instrument the JVM with the Quasar agent, and that the code modifications are coupled to a specific technology. It also only works with Java at the moment.

Watch this space!

1. Usually you want to avoid these, but for various reasons you may be compelled to do so

2. I’m assuming you have at least a passing understanding of how to write code in Vert.x)

3. In the meanwhile the latency of waiting tasks is shooting up and work may have to be rejected via timeout.

4. Or making progress in some meaningful way, such as streaming data.

5. And reducing responsiveness if/when the threadpool is exhausted.

6. Switching stacks and registry values; lots of thread state changes; etc

7. There are more advanced options available, such as multithreaded workers which can help improve performance on specific workloads, but are out of the scope of this blogpost. Multithreaded workers require thread-safe code.

8. I call this the event bus pattern for blocking code.

Byteman now available in homebrew

Marc Savy — Fri, 16 Sep 2016 00:00:00 GMT

A friend and colleague of mine, Andrew Dinn, is the brains behind the brilliant Byteman project. I recently contributed a brew formula to install Byteman and all of its scripts on OS X systems - allowing a package manager to sort out installing everything in sensible locations, with upgrading and other hassles handled for you.

Install brew if you haven’t, and then run ^[1]:

$ brew install byteman

If you need to, you can install the latest development version of Byteman with (e.g. ALPHA releases):

$ brew install byteman --devel

You can upgrade Byteman with ^[2]:

$ brew upgrade byteman

You should now have all of the Byteman wrappers and binaries available on your path!

$ bmjava -version

What’s Byteman?

If you’re not familiar with Byteman, it’s an incredibly powerful byecode manipulation utility for testing, monitoring and tracing. It can be invoked via scripts or API.

It’s particularly useful for tracking down difficult-to-reproduce concurrency bugs, as you can use it to precisely orchestrate thread orderings (interleavings, etc), wait periods, inject faults etc. There are also some cool up-and-coming projects using it to dynamically inject tracing and monitoring into existing code.

1. You may need to brew update to download the formula if you haven’t done so in a long time.

2. Or just brew upgrade to do everything.

Changing the root-context for apiman's gateway

Marc Savy — Wed, 24 Aug 2016 00:00:00 GMT

An apiman user recently asked me how they could change the root context of apiman-gateway from /apiman-gateway/ to something else, for instance /i-love-apiman/.

If you’re using a load balancer or some other mapper, just skip to the Alternatives section or read the apiman production guide.

If you are using the WildFly or EAP apiman distributions (such as our quickstart), here’s an easy answer ^[1]:

Unpack apiman-gateway.war.

$ unzip apiman-gateway.war -d /tmp/apiman-gateway

Edit /tmp/apiman-gateway/WEB-INF/jboss-web.xml to be to your new preferred context, e.g. i-love-apiman.

Repack the war.

$ zip -r apiman-gateway.war /tmp/apiman-gateway/META-INF /tmp/apiman-gateway/WEB-INF

Replace the existing deployment.
Alter your apiman.properties file and set apiman-gateway.public-endpoint=. ^[2]
```
apiman-gateway.public-endpoint=https://localhost:8443/i-love-apiman/
```

Result

When you publish an API to the gateway setup you modified, you should see something like this in the UI:

Alternatives

You can, of course, just set mappings on your load balancer (or some other technique external to apiman) if you want to avoid fiddling with the deployments. Even then, you’ll still probably want to set apiman-gateway.public-endpoint to report a sensible value for your setup.

One final note: you can set these options via Java’s system properties, too

$ ./bin/standalone.sh -Dapiman-gateway.public-endpoint=https://localhost:8443/i-love-apiman/

1. This procedure would be slightly different for other platforms such as Vert.x, Tomcat, etc

2. See: http://www.apiman.io/latest/production-guide.html#_setting_the_api_gateway_public_endpoint

Beware the Connection Header in HTTP/2

Marc Savy — Tue, 23 Aug 2016 00:00:00 GMT

Whilst tinkering apiman’s Vert.x gateway implementation I ran into a strange issue when invoking an API via curl --http2. It seemed to work fine in Chrome and using Go’s h2c:

$ * http2 error: Invalid HTTP header field was received: frame type: 1, stream: 1, name: [connection], value: [keep-alive]

This is what the setup looks like (distilled):

HTTP/2 Client <--> Apiman Gateway (HTTP/2) <--> Test API (HTTP/1)

What’s going on?

I took to Twitter to figure out what was going on.

@apiman_io @clementplop although the cURL --http2 doesn't like the headers Vert.x sends - interesting https://t.co/0dK81huCmb
— Marc Savy (@marcsavy) August 20, 2016

Out of the blue, Mr cURL himself (Daniel Stenberg) popped up with the answer quoted from RFC 7540:

@marcsavy @apiman_io @clementplop "An endpoint MUST NOT generate an HTTP/2 message containing connection-specific header fields"
— Daniel Stenberg (@bagder) August 20, 2016

Mystery solved

Aha! The backend API apiman was proxying the response from returned a Connection: keep-alive header. Apiman simply forwarded this on, but it turns out this violates the HTTP/2 spec as there should be no connection-specific header fields - in response cURL spits out the error we see above. ^[1]

HTTP/2 Client (http2 error...) <- Apiman Gateway <- Test API (sets "Connection: keep-alive")

The solution was just to filter out Connection headers in apiman, but I wonder whether Vert.x should do it by default (or show an error message, exception, etc).

That said, I think "it’s the app’s responsibility not to do stupid things" or "we should handle it and filter out Connection headers" are both valid responses depending upon philosophy.

So, I let the Vert.x guys know, and let’s see.

1. Other implementations seem to just ignore these irrelevant fields.

Setting up Vert.x HTTP/2 with JKS

Marc Savy — Sun, 21 Aug 2016 00:00:00 GMT

I was playing with Vert.x’s excellent HTTP/2 functionality with the aim of determining how difficult it’d be to add support into apiman's Vert.x gateway implementation (which is still somewhat of a tech preview, but it’s finally getting some love). Many HTTP/2 clients don’t support non-TLS connections, so transport security is a de facto requirement in the real-world usage (particularly with browsers).

There’s already an example in the Vert.x core-examples repository demonstrating how to set up and use HTTP/2. However, when using a keystore in JKS format, Vert.x (in v3.3.2, at least) tries to use Jetty’s alpn-boot ALPN implementation rather than OpenSSL.

A few simple steps…

So, here’s how to go about making it work:

Download the version of apln-boot corresponding to your version of Java. The binaries are available on Maven Central, or you can build your own if you prefer.
When launching your verticle, put alpn-boot on the JVM’s boot classpath; it’s altering core library functionality and needs to be available early. There are a few ways to do this, here’s the easiest one:
```
$ JAVA_OPTS=-Xbootclasspath/p: vertx run Example.java
```
If you’re using an IDE you should add -Xbootclasspath/p: to the VM Arguments of your run configuration.
Set setUseAlpn(true) on your HttpServerOptions.

Testing it out

Here’s some code to test it with.

import io.vertx.core.AbstractVerticle;
import io.vertx.core.http.HttpServer;
import io.vertx.core.http.HttpServerOptions;
import io.vertx.core.net.JksOptions;

public class Example extends AbstractVerticle {

    @Override
    public void start() throws Exception {
        HttpServer server =
        vertx.createHttpServer(new HttpServerOptions()
            .setUseAlpn(true)
            .setSsl(true)
            .setKeyStoreOptions(
                new JksOptions()
                .setPath()
                .setPassword()
            )
        );

        server.requestHandler(req -> {
            req.response().putHeader("Content-Type", "text/plain")
                .end("Greetings, Earthicans!\n");
        }).listen(8443);
    }
}

You’ll likely want to check that it’s really working. I’ve been using cURL compiled with nghttp2 to enable HTTP/2 support; depending on your OS or distro, the shipping version may have been built with it. If you’re using OS X, you can use brew install curl --with-nghttp2. ^[1]

 $  curl -k -v --http2 https://localhost:8443/
 *   Trying ::1...
 * Connected to localhost (::1) port 8443 (#0)
 * ALPN, offering h2
 * ALPN, offering http/1.1
 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
 * successfully set certificate verify locations:
 *   CAfile: /usr/local/etc/openssl/cert.pem
   CApath: none
 * TLSv1.2 (OUT), TLS header, Certificate Status (22):
 * TLSv1.2 (OUT), TLS handshake, Client hello (1):
 * TLSv1.2 (IN), TLS handshake, Server hello (2):
 * TLSv1.2 (IN), TLS handshake, Certificate (11):
 * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
 * TLSv1.2 (IN), TLS handshake, Server finished (14):
 * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
 * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
 * TLSv1.2 (OUT), TLS handshake, Finished (20):
 * TLSv1.2 (IN), TLS change cipher, Client hello (1):
 * TLSv1.2 (IN), TLS handshake, Finished (20):
 * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
 * ALPN, server accepted to use h2
 * Server certificate:
 *  subject: C=lo; ST=localhost; L=localhost; O=localhost; OU=localhost; CN=localhost
 *  start date: Aug 20 15:23:02 2016 GMT
 *  expire date: Nov 18 15:23:02 2016 GMT
 *  issuer: C=lo; ST=localhost; L=localhost; O=localhost; OU=localhost; CN=localhost
 *  SSL certificate verify result: self signed certificate (18), continuing anyway.
 * Using HTTP2, server supports multi-use
 * Connection state changed (HTTP/2 confirmed)
 * TCP_NODELAY set
 * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
 * Using Stream ID: 1 (easy handle 0x7fb8aa008800)
 > GET / HTTP/1.1
 > Host: localhost:8443
 > User-Agent: curl/7.50.1
 > Accept: */*
 >
 * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
 < HTTP/2 200
 < content-type: text/plain
 < content-length: 22
 <
 * Connection #0 to host localhost left intact
 Greetings, Earthicans!

Hooray!

Vert.x really is a gratifyingly well-thought-out and easy-to-use.

1. But remember cURL is not linked by default so you’ll have to dig into its Cellar (e.g. /usr/local/Cellar/curl/7.50.1/bin/curl). Of course, you could alias it, or force link :-).

The Musings of Marc

Setting Content-Type for multipart/form-data values

Farewell, Red Hat friends!

I was involved in a shocking incident and want to talk about mental health - and what we're doing as a country to address it.

Follow-up

Introducing asciidoc-admonition-icons, a GitBook plugin to restore font icons for AsciiDoc admonition blocks.

Plugin to the rescue.

Installation & Configuration

Generic JWT policy for apiman

Build and install

How to use it

Apiman's New Execute Blocking Component

Blocking is bad, but…​

Execute Blocking

Apiman on blocking platforms

Final Words

Stopping Vert.x BlockedThreadChecker exceptions during interactive debugging

What’s the cause?

Workarounds

Via System Properties

Directly in VertxOptions

Caveats & Conclusion

Vert.x and Blocking Code

Why is blocking bad, anyway?

Solutions

Worker Verticles

Offering a blocking service

Consuming a blocking service

Execute Blocking Statements

Vert.x Sync

Byteman now available in homebrew

What’s Byteman?

Changing the root-context for apiman's gateway

Result

Alternatives

Beware the Connection Header in HTTP/2

What’s going on?

Mystery solved

Setting up Vert.x HTTP/2 with JKS

A few simple steps…​

Testing it out

Blocking is bad, but…

A few simple steps…