AWS released the new R4 instance type in late November, so you know I’ve got to test it with Tableau Server just as soon as I possibly can. For me, that’s after re:Invent.
The new R4 features 2.3 GHZ v4 Broadwell CPUs, as well as improved networking and throughput to EBS. Sounds like my sort of instance.
I stood up a 16 vCPU (8 core) R4 (r4.4xlarge) with 122 GB of RAM. I also baselined a similarly sized C4, M4, and R3 using the same workload.
In all cases, I installed Tableau 10.1.1 on its own volume – an EBS GP2 100GB disk.
|Instance||Cores / CPU||RAM|
|C4||8 (16 vCPU) E-2666 v4||30 GB|
|R3||8 (16 vCPU) E-2670 v3||122 GB|
|M4||8 (16 vCPU) E-2686 v4 or E-2676 v3||64 GB|
|R4||8 (16 vCPU) E-2686 v4||122 GB (DDR4)|
Since we now deploy 4 VizQl processes on an 8 core by default, I went with it. However, I also repeated each test with only 2 vizqls – our “older” guidance. I’ll ruin the surprise right now and point out that for my workload, 2 vizqls seemed to be better.
The workload itself consists of 83 different vizzes. Some are the Tableau samples we know and love from years gone by. Others purposefully do “expensive” things like:
- Plotting a total of 10M marks across 10-15 dimensions and adding reference lines
- Rendering a 1M+ mark bar chart
- Forecasting against many (20+) dimensions in a small multiple layout
- Executing against “large-ish” datasets (100M+ row extracts), file sizes between 2-3.5 GB
All in all, it’s fairly similar to workloads I’ve played with in the past. Aggressive, but maybe a touch lighter based on the results I saw across all instance types.
Let’s get at it, shall we?
The standard disclaimer
As I usually do, I’m hiding values on the axes so a few of you (you know who you are) don’t rat hole on the numbers themselves. The numbers only are relevant to MY workload, which you won’t ever run. I just want to show you how one type of machine performs against another…and then you need to test for yourself. As always, I used TabJolt for these tests. I applied 3 seconds of think time after a “view” and 5 seconds of think time after an “interaction”. 60% of my executions were “view only”, and 40% were “view, then interact”.
C4: Good CPU + BAD RAM = Not Great for Tableau.
No surprise, the C4 was the laggard. Tableau recommends 8 GB of RAM per core, and this sucker has slightly less than half of that at 30 GB.
You’ll see how poorly these performed against other machines when I put them altogether, but note the very HIGH error rate when I ran 4 vizqls. Once one of these 4 vizql machines was saturated it was really, really done:
Last Generation R3
The (now older) R3 also has122 GB of RAM, and a respectable processor. As you can see, 4 VizQls on this machine also had a higher error rate and response time than 2 VizQls. Transactions per second (TPS) were lower and the 4 VizQL machine seemed to saturate a little bit more quickly.
Here, we compare against the C4. The R3 clearly is the winner across the board.
Current Generation M4
My old beloved go-to instance, the M4. Here’s what it looks like, 4 VizQLs vs. 2 VizQLS. There wasn’t as much of a difference here.
Here’s the M4 against the R3 you just read about:
It’s pretty close to a dead heat.
As one would expect, the M4 also spanks the C4 with great, vigorous swats:
R4: The star of the show
And now, what may be the new heir apparent: The R4. With more RAM, we see this instance revert to the same pattern we saw earlier with the R3 – 2 VIzQls giving us lower response time, slightly higher TPS and much lower error rate (although peak error rate towards the end of this test is only 3% anyway, so who cares):
Against the venerable M4? It’s better. You can see this machine is able to handle more users, and gives us ~20% more TPS. Response time on the 2-VizQl machine is also lower than vs. the M4. I think we may have a new champion here!
Against the previous version of the R3? Better, of course:
So that’s it. The R4 is a pretty awesome instance for Tableau, and at only a small price premium over the M4, it’s something you should add to your test matrix and check out
If you want an interactive version of the vizzes above, you can find the dashboard in question here:
(If you’ve really got time on your hands you can tease out “actual” numbers via tooltips, as well…but remember, it’s NOT about the numbers – they are based on my workload, not yours).