Camunda Platform 8 for Camunda 7 users

In this special episode, Developer Advocate Niall Deehan sits down with Camunda Co-founder and Chief Technologist Bernd Ruecker to discuss the newly released Camunda Platform 8 and what users of Camunda Platform 7 need to know.

A few questions that are addressed in this episode:

• How is Camunda 8 different from  Camunda 7?
•  What inspired this next generation of Camunda Platform, powered by Zeebe? 
• When does it make sense to move to Camunda Platform 8?

If you want to find out more about Camunda Platform 8 from the experts or be among the first to try  Camunda Platform 8 in an interactive Code Studio session, you'll want to attend Camunda Community Summit on April 26-28, 2022.

For more details and to register, visit the Camunda Community Summit website. 

---

Visit our website.
Connect with us on LinkedIn, Facebook, Mastodon, Threads, and Bluesky.
Check out our videos on YouTube.
Tweet with us.

---

Camunda enables organizations to orchestrate processes across people, systems, and devices to continuously overcome complexity and increase efficiency. A common visual language enables seamless collaboration between business and IT teams to design, automate, and improve end-to-end processes with the required speed, scale, and resilience to remain competitive. Hundreds of enterprises such as Atlassian, ING, and Vodafone orchestrate business-critical processes with Camunda to accelerate digital transformation.

---

Camunda presents this podcast for informational and entertainment purposes only and does not wish or intend to provide any legal, technical, or any other advice or services to the listeners of this podcast. Please see here for the full disclaimer.

Transcript

Josh Wulf (00:10) : 

Welcome to this special edition of the Camunda Nation podcast, published for the release of Camunda Platform 8! It’s not every day that we get a new major version of the Camunda Platform. I’m excited, and I’m your host - my name is Josh Wulf, I’m a Developer Advocate at Camunda, and in this episode we’re listening in on a conversation between Niall Deehan, one of our Developer Advocates, and Bernd Ruecker, Camunda co-founder and chief technologist. 

In this conversation, Niall brings the perspective of deep experience with Camunda 7 and Bernd explains Camunda 8. This is a great conversation for anyone familiar with Camunda 7 to find out about Camunda 8. 

“What are the main differences?” 

“What are the main drivers for the changes?” 

“When does it make sense to make the shift?”

These are the kinds of questions that this conversation addresses.

So without further ado, let's roll the conversation between Niall and Bernd!

Niall Deehan (01:11):

Hello, this is the Camunda Nation podcast and I'm Niall. I have a guest with me today that has been inside the depths of Camunda 8 for a very long time now. Since its inception, I would say. Bernd, would you like to introduce yourself?

Bernd Ruecker (01:24):

Hey Niall, great that you have me today. My name is Bernd Ruecker, I'm one of the co-founders of Camunda. I was at the company forever, obviously, and I've worked on a couple of different open-source workflow engines in the past. I was part of the jBPM team. Then I was also part of the activity team, and part of Camunda 4, which became Camunda BPMN, which now becomes Camunda 7. And I looked into a lot of technical architecture. And that's actually my thing, looking at architectures, looking at problems at customers, looking at how our technology basically can solve some of the problems at the customer’s site. And I was looking a lot into distributed systems and microservices over the last 5, 6, 7 years. And also at how to architect engines that solve these problems properly.

Niall Deehan (02:13):

And we will definitely be covering a lot of architectural questions today, because that is a fundamental direction that Camunda 8 has chosen. But before we actually go into all of that, there is a whole bunch of terms we're going to be throwing around over the next 40-odd minutes. I want to just define some for people. Camunda 7, going forward, as we discuss it, will be essentially the product that Camunda started with in 2013. It's a Java engine, it's embeddable, it's centralized, it has a relational database.

Niall Deehan (02:48):

It's the thing that we've all been using for many years. Cockpit and task list would be included in Camunda 7. When we mention that, that is what we're talking about. Now Camunda 8 is also a new thing. I'm not going to define that, because I have someone far better qualified to do that. Bernd, would you like to give me a quick definition of what Camunda 8 is?

Bernd Ruecker (03:08):

Happy to. So we introduced Camunda Cloud a couple of years back already, as the cloud native environment to run workflows and Zeebe was the workflow engine. And with Camunda 8, we basically repositioned Camunda Cloud, not as a parallel product offering, saying, "Camunda 8 is the next product, the next version of the workflow engine."

Bernd Ruecker (03:32):

And you now get it in two flavors, which is either hosted by us, which is also called SaaS, or Software as a Service. Or this is normally what you refer to as cloud. So, we host the workflow engine, you can just provision a new one from us via a web console, and then connect to it. That's a SaaS offering. Or you host it yourself on your own premises, or in your own data center, or at your own cloud provider, which we call “self-managed”. That's the self-managed flavor of Camunda 8. And both include always Zeebe as the workflow engine.

Niall Deehan (04:05):

Zeebe is one of the main components of Camunda 8, but there's also some other components, because there's other tools that people probably know about in Camunda 7, like Cockpit, for instance, and Tasklist. The equivalents do exist in Camunda 8, right?

Bernd Ruecker (04:20):

100%, some of the tools are in a way, unchanged. If you look at Optimize for example, the business intelligence tool that looks into all the process data, it's exactly the same. It can be fed with data from Camunda 7 or Camunda 8. We have a different task list, it's still called Tasklist. It's a new one which basically connects to Zeebe. We have a different architecture underneath, which we might talk about later. And then there is a tool called Operate, which is the equivalent to Cockpit.

Niall Deehan (04:48):

Thanks Bernd, from here on, I'm going to throw some questions at you. Probably some of the questions that I would've had when I first heard internally about the change in name. The first one, is I'm sure the one that any big fan of Camunda 7 would have, which is, is this death of Camunda 7? Is Camunda 7 over? Is that lovely era of having a relational database over? Can you talk a bit about that?

Bernd Ruecker (05:14):

This is the question that obviously has to come up. That's part of saying, "This is Camunda 8. This is the next version." Of course everybody goes like, "What's with Camunda 7? What's with the existing version? Do I have to migrate? What's the time window?" And so and so forth. There are a lot of really deep changes in the way the workflow engine works. And this is a big change actually. We can explain that later on, why we are doing that, and what are the great advantages of that. But what we also see is that it's not like with the click of a button, you migrated from 7 to 8.

Bernd Ruecker (05:47):

We don't expect our customers, our users, to actually do that. The current assumption is that if you start a new project, you might go to 8 straight away. But if you have existing projects in Prod, or probably you have your development almost finished, and want to go production anytime soon, you can, of course, continue with Camunda 7. And we defined a guaranteed, we call it full-support time cycle for the next five years. That's quite some time. And during that time, we will not only do critical security bug fixes, but we will also do additional features, maintenance. You know it, we basically don't change much internally. We have the same development team continuing working on Camunda 7.

Bernd Ruecker (06:32):

Today, we also have contracts that go beyond that. I expect that we have at least some maintenance support for quite a long time. It is a successful product. It's out there in a lot of environments, so don't expect it to die anytime soon. And we might also discuss that a little bit later. But there are a couple of use cases where I still think Camunda 7 is a great choice, even for new projects. And then go for it, that's no problem.

Niall Deehan (07:01):

That makes sense. We're basically going to have the idea of - similar to a lot of software companies, where you release your new version, but you want to keep the people who are using the old version still there. But we also need to indicate to brand new users, that Camunda 8 is the successor, to a certain degree, so anyone coming to Camunda will know, "Okay, well hey, there's Camunda 8 and Camunda 7. I think I'll probably try Camunda 8 as a first run through." The majority of our community are pretty much untouched by the number change for now.

Bernd Ruecker (07:32):

100%, and I think it even makes it easier for the community. At the moment we have two products, or at the moment. It was before we basically went to Camunda 8. We had Camunda Cloud and Camunda Platform. And this was, I think, more confusing for people, because then you see two products, and you have to choose between them, so you have to understand where are the differences? What are my needs? A lot of things, and that's not super-easy. It's an obstacle to get started. And now it's much easier, you go Camunda 8, this is the version you do.

Bernd Ruecker (08:03):

But there might be reasons to still go for Camunda 7, which is the exception. But then, you normally know what you're doing. And that could be because you already started, or because you're already invested in that. And there are a couple of reasons why you might want to go with Camunda 7. But in this case, you know why. You don't to discuss that upfront.

Niall Deehan (08:21):

And I totally agree. I think that's been a problem for a while, where people have been unsure of what cloud means versus 7. And this is really, I think, I do agree, it clarifies things for the community.

Bernd Ruecker (08:34):

And just to add one comment, what we saw very often and we named it Camunda Cloud, because it was the thing that's cloud native as a SaaS offering provided to people. But we had one use case, which regular comes up. If we discuss Zeebe in the past, very often we talked about horizontal scalability, about mass throughput, a lot of process instances, like probably millions per second, if you like. And this was the one use case, which was always tackled by Zeebe, even like two or three years back.

Bernd Ruecker (09:05):

We have big customers, big projects going for Zeebe with these kind of requirements, but very often this was not SaaS, this was self-managed. This was on-prem. These were very critical projects, probably financial industries, so they wanted to have it under their control. But the term Camunda Cloud always included that you have to run it in the cloud, it must be SaaS. And it was like, no, no, you can have Camunda Cloud self-managed, it was confusing as well.

Niall Deehan (09:35):

And we'll talk about the two differences a little later on, but actually now that we're on the key point, which is the differences and the different use cases. Camunda Cloud and Camunda 7 are architecturally, fundamentally different, for sure. Some things are implemented from the ground-up. And I know from my own messing around with cloud, was a lot less extensive than yours. That some of the things I wished were implemented a certain way in Camunda 7, are implemented in Camunda 8. And we'll get to some of those in a little while in more details, like message events and how they've changed. And multi-instance parallel and stuff. But before we get to all the real details, can you give us just a high level description of the core differences of the two platforms?

Bernd Ruecker (10:24):

We can dive into any depth here. And there are a couple of different interesting things about it. But if you look at operating the workflow engine from a high level, the difference is, first of all, we don't need a relational database. Camunda 7 basically bases all the data access to a relational database, Camunda 8 or Zeebe does, it's kind of an event sourced approach, so basically it stores everything itself on disk, and does replication, and a couple of other things itself. We are not relying on a relational database.

Niall Deehan (10:57):

That is legitimately, I think the one thing people really know about Zeebe. We made it very clear, I think, from the beginning that one of the things we wanted to get away from, was this relational database, that 7. Can you just maybe talk to why that was an important thing to want to move away from?

Bernd Ruecker (11:17):

I think there are a couple of good reasons for doing that. One of the big drivers was resilience and scalability, actually. Honestly this was one of the starting points where we really thought about, Hey, how can we do cross automation technology in a more scalable way? When we were at a customer, it was a financial institution, and we did a cross automation project for, I think it was onboarding. Onboarding is a pretty common use case. You onboard a new customer and they get probably whatever, a new bank account, a new credit card or whatever. This was an onboarding process and you run that if you're a big international bank, you run that regularly. You are in the multiple, thousands per day, but you're not in the millions per second. And then this was great. Everybody was super-happy with that.

Bernd Ruecker (12:05):

And then they basically asked like, "Hey, we all these great advantages, can we use that for the trading process over there?" And it was like, "Oh, that sounds interesting. How much trades do you do?" And I got a number, it was like, whoa, "Probably don't do this." Because you know that the relational database is always the bottleneck in the architecture with a relational database. And that means you can scale it to some extent, and it was surprising how far we could get with that actually. But at that point in time, at some moment, you're stuck. You hit a ceiling. And then you can do other things like charting on top of the engine, like logically running multiple engines, and so on and so forth. But it's always a pain and you don't get beyond that.

Niall Deehan (12:52):

Also, I have a very clear recollection that you just mentioned, like going to a customer and hearing this essentially, "Hey, can we use it in this way?" And having to say no. And it reminded me of going to a large telco company, back when we did consulting. And trying to talk through how to implement georedundancy with Camunda 7. They said, "Hey, we want to have this run in two different data centers. And we want them to be active active." And I remember thinking, what an absolutely intense amount of time we had to spend on trying to think of how I can make this work. I guess, now thinking back, without the database, and we've seen this with Camunda 8, this becomes not trivial necessarily, but certainly-

Bernd Ruecker (13:43):

It gets way easier. And especially, way more stable. It's a very clear pattern in how you do this. And I might remember the same product you have in your head, but I remember we had a MySQL, and we even did attempts to replicate that. And geo redundant between continents, you have a lot of latency between these two databases, and that adds up. And the problem with the relational database architecture, is that this added latency blocks a lot of open transactions, blocks a lot of threads, and then at the very end, you end up with a system which can't do much anymore.

Bernd Ruecker (14:20):

And this is very different now. And we have already a customer in production using geo redundancy for their system. And you still have the latency between two data centers. But what happens now, is that the throughput, how you process stuff is still the same, but for every process instance, you add that latency, because you have to replicate. Yes, I mean, you can't do magic, you have to wait for this. But it doesn't affect throughput at all. And this is a very different architecture from the ground up. For resilience, and also performance scalability, that's the second thing, we can now scale horizontally. We proved that in a couple of experiments now. That was one of the core drivers.

Bernd Ruecker (15:02):

The second thing that also motivated us to dive deeper, is really that we see that software is built differently than we did it 10 years ago. There were things like, obviously microservices, serverless, a couple of other things, that led to what's now summarized as cloud native in a way. I build an application that naturally leverages certain capabilities which are in the infrastructure, like a database, but probably also others. And it got so much easier now to start up some resource you need, like locally you can use a Docker container, or use a cloud environment, and just kick off a service. It got way easier to just provision a workflow engine, and use it as an external resource to your application.

Bernd Ruecker (15:51):

And then you make the distinction very clear, what's my application code, what's my business logic in a way, and what's the technical resource I need. And there are a lot of advantages of actually doing this. And by now, we can actually do that in a very convenient way. And I think this all changed a lot over the last five years, which also enabled to make this move now.

Niall Deehan (16:16):

And I do think that it's not just that the change happened without Camunda 7 being involved. We know ourselves that one of the most used distribution of Camunda is Camunda run, which is a centralized engine that is basically treated like a service. And actually, that probably goes to speaking about another very key difference about 7 and 8, which is how the stack works. Because in Camunda 7, there are lots of ways, I would argue too many ways of using, which meant we implemented lots of features, for lots of different ways of using it. And I think Camunda 8 is more focused. And as you said, it's focused based on current modern architectural stacks. Can you talk a little bit about the current patterns and architecture that we expect people to use with 8?

Bernd Ruecker (17:09):

And this is exactly right, if you're looking at Camunda 7, and you know Camunda run an external task, this is more or less like the architecture also works if you're using Camunda 8. You're having Zeebe as the engine, it runs somewhere. It might be locally on your Docker container, if you're just playing around as a developer. It might be in the cloud managed by Camunda. It might be somewhere in your own, like self-managed in your data center. If I say your data center, I normally mean the cloud of somebody else.

Bernd Ruecker (17:40):

This is kind of the model, and then you build your application code. And this can be exactly the same as in Camunda 7. It might be a Spring Boot process application for example. That's actually the model we see most. It might contain the BPMN file, and it might contain some of what I call glue code, which basically glues your process model to, for example, a service call. All these pieces of code you need in order to do that. This is the way how you do it in Camunda 8. In Camunda 7, there were a couple of other ways as well, but now this is the one and only way. The big advantage we get now, is that this external task pattern also allows to connect to the workflow engine from different languages for example, programming languages. We are using gRPC for communication underneath. You don't really see that as a developer using Camunda 8, because you have a client library, an SDK you use. And we have that for Java, we have that for Node.js, there's Python, there are a couple of others in the community.

Bernd Ruecker (18:40):

You can generate the language code in any language you like, so this opens up a lot of new possibilities actually. And a lot of new ways of adopting it in a polyglot way. Which I find super-great, because in the past we had, "Hey, there's the 15 ways you can use the workflow engine" but if you're not Java, you're out of the game.

Niall Deehan (18:58):

Yeah true. I also like the idea that it focuses us internally. I think of our development team. When you get a certain feature request, for instance, for Camunda 7, you need to think which users, in which type of architecture does this benefit? And it takes time to answer those questions, and to assess whether a certain feature will benefit enough users. By streamlining, I guess, Camunda 8, it means, at the very least, we have a single way to use the engine, which means features are going to affect everybody who uses the engine in the same way.

Bernd Ruecker (19:32):

Very true, and I think the more extreme way of seeing the same, is actually also on support cases. If you have problems that it takes really a lot of time to understand what's the exact environment the customer is using, how did they use the engine exactly? How do they wire transaction management? How do they configure it? What plugin did they add there? There was no isolation at all between the application code and the engine. And this consumed a lot of time. And not only on our side, also on the customer side, because there were too many options. And I totally agree, now we have a very clear way. This is how you run the engine, and this is your code, there is the interface. And this makes pinning down problems much easier.

Niall Deehan (20:18):

Now we've discussed that basically Camunda 7 is not going anywhere. It's going to be around for a long time, and people should be pretty comfortable with maintaining that. We also discussed that there are some fundamental differences in the two architectures. I think it's very important we discuss why someone would want to move to Camunda 8. Or why they would choose it over Camunda 7, given they have both options. Especially from the perspective of someone who already uses Camunda 7. What are the advantages there? Why would I want to use Camunda 8?

Bernd Ruecker (20:50):

I would differentiate two different scenarios. One is like, you already use Camunda 7, and you have something in production. If that's the case, and if you don't have any concrete problem or pain, I actually would keep it running. There's no immediate value you get from migrating to Camunda 8. I think the other scenario is, you have some problems. For example, I talked to a customer last week, and they do millions of process instances per day on a Postgres database. And they have a couple of issues with deleting data, like in certain timeframes, indices, so they start looking into optimizing performance issues on Postgres. And that's the moment where I say, "Hey look, probably it's better to invest this effort now in looking at migrating to Camunda 8, because we don't have these problems there, and we can easily prove that, instead of optimizing Camunda 7 for performance." That's one example where I say, "Okay, look better look for migration." And like what we said earlier on, if you start a fresh product, I would simply go with Camunda 8, because this is where a lot of innovation also happens.

Bernd Ruecker (21:57):

We still invest in Camunda 7. I'm pretty sure we will still build a lot of features there. But for example, for Camunda 8, we're looking at the whole developer experience in a more holistic view, hey, how can we get to what we call [inaudible 00:21:59], to your local development environment? How you can interact, collaborate with people in the web? How can you easily deploy to a managed workflow engine? How can you connect that for example, to things from Operate or Optimize? A lot of ideas are currently happening there. The same goes for connectors, Camunda 7 users might always be scared of the term connectors, because in the past, we kind of failed on delivering really good connectors. And we even recommended in the best practices, "Probably don't use connectors."

Niall Deehan (22:47):

I think the forum is full of responses by both of us saying, "Just use external tasks, they're just better. There's too many flaws." Connectors have the unfortunate side effect of being very easy to implement from nothing, and subsequently can't scale.

Bernd Ruecker (23:06):

Yeah, but we learned from that in a way. That's one thing. And also the other thing is, we are listening to customers over the last couple of years, what's the thing they are missing? And just to sketch one scenario what we see regularly. When we are in bigger corporations, they very often have normally a center of excellence, part of the organization, where they look at process automation projects. And very often, they help internal projects to be more efficient, or faster to implement, or automate processes.

Bernd Ruecker (23:42):

And what they all tell us is that, they have certain pieces of code they would love to reuse across the organization, because they have to talk to, for example, their internal mainframe system somehow. Which is not super-easy to go, and there is not 100 different out of the box, open-source products you could simply pull in. It's not super-simple. Or there are certain things when you, even connecting to your whatever, SharePoint, or whatever it is. This is the common theme that they would love to have some kind of reusable building blocks, which then the developers that pull together the automated processes, the workflows, can configure and reuse.

Bernd Ruecker (24:26):

And this is what we are now looking into, how can we make that a really painless, a great experience. Different from what we saw with Camunda 7, that you have the part in the modeling UI, that you have the part in the code in order to make a connector. To have the part that makes it possible for companies to develop their own connectors and share that within their ecosystem. And this is something we're really looking into quite heavily at the moment. We will start also with a handful of connectors, but it's the start. We're investing a lot in there.

Bernd Ruecker (25:02):

And this for example is an innovation that's happening in Camunda 8. And you will not really benefit from that in Camunda 7. Which again, doesn't have to be a problem. I mean, if you have a running solution, it still runs. You don't miss the connectors probably.

Niall Deehan (25:17):

Now you did mention that, and it's a very important point, which is if you have no problems running, that's fine. But there's also another angle to this, which I found myself, which is, if there's stuff that's really irritating, there can often be a much easier solution. Some examples that you mentioned, just like for instance, being able to share code across things. But also, I found myself upgrading your Camunda 7 instance, if it's a very big, like upgrading the version of Camunda in a very complicated architecture, is a lot of work. And you need to get someone from DevOps to properly run the scripts, and you need to update the artifacts, redeploy stuff. You need to rerun testing, and so on, and so forth. I know from the messing around I've done with Camunda 8, that there's a little button that says "Upgrade."

Bernd Ruecker (26:03):

Yeah, for us running the SaaS version, it is essential that we can update stuff in a rolling fashion. For example, for trial accounts, I think we even do that automatically, so you can't even choose. It's automatically updated. It's a trial account, I mean, hey, it should have the latest stuff. For paid users, or production systems, we don't do it automatically, but with the push of a button.

Niall Deehan (26:32):

And that is again, one of those things that people might have been struggling with in Camunda 7, that they can think, oh, that becomes much easier then.

Bernd Ruecker (26:33):

I know users that have built some kind of fleet management software to manage different Camunda 7 engines, at least to harvest all the versions that are running, in order to get an idea of what they need to update. And yes, this gets a lot easier.

Niall Deehan (26:55):

And you did mention the Camunda SaaS there, and I think one of the problems you mentioned at the very beginning, which I think we should talk about now, is Camunda Cloud was the name for Camunda 8, and the problem that caused on a messaging level. There are two flavors of Camunda 8, and one of them is cloud based, and one of them is self-managed. Can you maybe just explain how they work, and the differences if there are any? And why someone might choose one or the other, for instance?

Bernd Ruecker (27:28):

Basically how you operate Camunda 8, it's in a way, always Kubernetes. It's based on Kubernetes, so then you run, what's named Pots, which are basically virtual machines on Kubernetes. And then you have a couple of these components which are started up. For example, Zeebe, the workflow engine, normally you run at least three brokers, which then replicate the data, and share the work. And you have other parts which you might run. For example, we use Elasticsearch underneath Operate or Optimize. And then of course, you have these tools like Operate, Optimize, Tasklist, which you also run, which are own small Java Spring Boot applications.

Bernd Ruecker (28:09):

You have all these components, and basically you could start them locally. We have a Docker compose file for example, to start all the components locally on your machine. And then you can access that easily. It's meant for playing around. And this is something you can do if you have a powerful machine. You shouldn't do that on a tablet or something like that. Every Java application, it's not huge, but it needs a bit of memory and a bit of CPU, otherwise it doesn't work. This is for playing around.

Bernd Ruecker (28:38):

But for a production environment, we always base on Kubernetes, and then there are these two ways. One is like, you do it yourself, self-managed, and then we provide Helm charts. Helm is the way to go in the Kubernetes ecosystem. It basically describes which parts or which containers you want to run, what resources they should get, and how they are wired together, and so, and so forth. That should be a relatively easy thing to do. But you need a Kubernetes cluster at your disposal. And you should have at least some Kubernetes know-how in the company. You can definitely run that on a cloud provider. All the big cloud providers have a Kubernetes offering. That's one of the options how you can run it self-managed. And then you have it under control.

Bernd Ruecker (29:23):

And the second option is SaaS. We host it for you, and then you go to come on a cloud. You can use a social login or provide some data, and then you can immediately start a 30 days trial. And that means you're having a web console, where you can just like, "Hey, I want to have one new cluster." And then you get a new cluster. And a cluster means, you get the Zeebe engine, you get Operate, you get Optimize, and you get everything underneath that's needed to operate that. And Tasklist, I always forget about Tasklist, sorry. This is very easy. And then in the web console, you get links to the web applications, and you can create credentials. Basically you have an URL endpoint for the workflow engine and you get basically a secret, and you can copy that to your local development project. If you're using Spring Boot, you could also use environment properties or whatever, the way that is that you normally use. And then it can start up locally your application, and it connects to the workflow engine in the cloud.

Niall Deehan (30:21):

And we do have some examples already for people who want to get started. There are some pretty good projects that already exist on video tutorials, on how you can build your first process with Camunda 8. You've quite well explained in a lot of depth about the two different offerings, I'm interested, which one is the default? Which one should people try first? Not from a just get started perspective, but let's say from, I have already decided to use Camunda 8 for my project, what is the better option? Is it to go with the SaaS offering or the self-hosted?

Bernd Ruecker (30:54):

For me, the default would always be SaaS. Why? Because you don't have to even think about how to operate the engine. That's the whole cloud idea. Somebody, in this case, the vendor who knows the technology best, operates it in a way that makes sense for me. And I don't have to care about it, I simply use it. It's by far the easiest model. And you don't need infrastructure people for doing that. You don't need anybody on the call for 24/7, and so on and so forth. That's definitely the default, and that's where I think the world is heading. Then there might be reasons why this is not possible for you. Why you might want to make an exception and run it self-managed. And the typical thing is compliance. And we have a couple of customers, bigger customers, for example in the insurance or banking industry, which sometimes, only sometimes by the way, are not allowed to run it on our cloud offering. And then they simply do it self-managed.

Niall Deehan (31:53):

Yeah that makes sense. Now, you did mention running stuff locally, and probably that's a nice segue into discussing one of the perceived differences, but realistically, similarities between Camunda 7 and Camunda 8, which is the open-source status. For instance, Camunda 8 has an open-source process engine. It has a mostly open-source front end. It has a mostly open-source monitoring, and Camunda 8 is very, very similar with some caveats. Can you basically give me a better understanding of if there are differences where they are, and if it would affect people in general?

Bernd Ruecker (32:29):

There are basically, I would say two differences to the Camunda 7 stack. One is the core workflow engine, Zeebe is still, I would call it open-source. But there is a big discussion in the community if that's the case. We did what we call, or what's now known as, Source Available license. Basically what it does, it says, "Here's the source code, you can do with it what you want" in a way. But the only thing that is not allowed, is taking the product and hosting that as a SaaS solution. And that's the only thing that's forbidden by the license. Some people named that the anti-cloud vendor or anti-AWS license. And it was actually born around these discussions. For example, Elasticsearch was very, very famous in the press a couple of years back, where the AWS hosting of Elasticsearch got more common actually, than the Elasticsearch offering.

Bernd Ruecker (33:26):

And then as a vendor, this is a really problematic situation, because the monetization goes to the cloud vendor. But you are basically building the product. And this is why there is a lot of movement in the industry at the moment, to discuss this. And I think it's a very important discussion now, how can we keep open-source alive? I find it a very good model actually. And for us, it worked very well so far, and I love it in a way. But we also need to protect it. And this is why we went for source available. But it means for normal people, if you're not a big cloud vendor, it means it's still the same. You can do with it what you want. You can embed it, or integrate it in your solution, no problem.

Bernd Ruecker (34:11):

That's one difference, which was simply timing, because we developed the workflow engine from scratch, we could choose the license again, so we did that, because that was in a time where you already know, or knew the problems that you can cost. And the second difference is, I would call it, we cleaned up the open core model. With Camunda 7, we already had open core, so you could have Cockpit as an open-source component, so you got an operating tool. But it lacked certain features. And these features were only available in the enterprise version, so only in the paid version.

Bernd Ruecker (34:39):

From our experience, and I think you can talk to that very well, this led to a lot of problems in a way that, first of all, people probably didn't even know that these features exist, so they sometimes redevelop them from scratch, simply because they didn't know. Another problem what we had commonly, people were showing stuff in a video, in a tutorial, or on a conference talk. People saw it and were like, "oh, I want to do that. Hey, I don't have this button. Why? This is so confusing." It caused a lot of problems. We actually rethought that. And we now go with a model with upgrade, which is available, like all the features for everybody, so you can always download it, you can run it. But it has a license that doesn't allow production usage. It allows development, for example. You can do whatever you want with that in development. But as soon as you go production, you need a license. And I find that a far easier model actually, because then you see all the features. You can also test it in a way that, hey, does that make sense for me or not? That's the second difference. And for example in SaaS, that doesn't matter at all, because I mean you always have a license anyway.

Niall Deehan (35:59):

That's really interesting. I think the question that we should spend the rest of the next, maybe 10 minutes talking about, which is the question that I think we hear almost all the time, which is migration. The big M. Let's imagine that I have made the choice, as a C7 user, Camunda 7 user, that I would really like to use Camunda 8. And that is going to make me a happy person. Where do I start? What's the procedure? Is it possible that there's scenarios where I can't migrate? Or where it's easier? That sort of thing. Can you speak a little bit to that?

Bernd Ruecker (36:35):

First of all, there is a migration guide in the documentation, so that's basically a good starting point to look at that, because it describes not only the conceptual differences between the products, but also the concrete steps. What you have to do. And basically, I mean, it's not rocket science, there are two things you have to look for. The first is your process model, or by the way, your DMN model DMN is also, in a way affected, but the changes are not very big there.

Bernd Ruecker (37:02):

But your BPMN process model and your code. Looking at the BPMN process model, it's relatively simple. For example, it's still BPMN, so 90% or 95% is the same. The rest are vendor extensions that we used in Camunda 7. And they might differ slightly in Zeebe and Camunda 8. There is documentation for that. There's even a model or a plugin for our modeling tool, where it can convert models from Camunda 7 to Camunda 8. I would call that a best guess. It goes for the 80% of normal cases, and it leaves some room for things that might not be migratable.

Bernd Ruecker (37:42):

There are two reasons why. First, it might be that we simply haven't yet implemented that. That might change, it might improve over time. The second is there are always weird extensions, or hacky things people did, which might not be in the migration. But it should give you a good basis, and then you can either extend the migration tooling there, or you simply rework some parts of the model manually. That really depends on, do you migrate one process model or 1,000? But it's doable.

Bernd Ruecker (38:11):

And the second part is the code. And there are also two things about the code. One is, what we call client API, so always when I talk to the workflow engine, "Hey, start a new process instance." "Hey, whatever." This API changed, so you have to simply adjust your code to the new API. Again, depending on the amount of code, and how you encapsulated that, this might be easier or horrible. But it's not a super-complicated thing to do. It's simply work.

Bernd Ruecker (38:42):

The second thing is a glue code, so the Java delegates typically in the Camunda 7 world, or the expressions, the spring beans you're calling directly. And then there are two ways of doing that. We have an adaptor project, which basically can adopt the Camunda 8, let's say external task way of doing things. And still then invoke the Camunda 7 Java delegates, immediately that can work. Or the second option is, you rewrite your code as well. And basically let's say upgrade it to the latest safety directly. It depends a little bit on again, how much Java delegates do you have, how complicated are they, and so on and so forth.

Bernd Ruecker (39:25):

Both ways have advantage and disadvantages. The whole migration, I think, works for 80% of the cases relatively easy. And then there are some cases which simply are not easily migratable. And that's typically if you did integrate with a workflow engine very tightly. We had situations for people share transaction manager and do weird things with that. Or they use thread locals, and do weird things with that. Or they cast to internal workflow engine classes in the Java delegates to do weird stuff. All of these hacky things, they're not possible with Camunda 8 anymore, which I personally think it's a good thing to have. But of course, if you're in a migration situation, and you have a very clever workaround for a very specific problem, this might be a problem.

Niall Deehan (40:21):

And there is also some things that, for instance, would've been hacking workarounds that are no longer necessary, that are fixed in Camunda 8. For instance, I know loads of people who have had to have all sorts of workarounds in order to ensure BPM messages don't get lost. And we know that now, and when you build a workflow from scratch, like Camunda 8 and Zeebe, we can basically say, "Hey, we can see the problem with the BPM specification in particular when it comes to BP messages" and we can just build in a queuing mechanism, or a time to live mechanism for messages.

Niall Deehan (40:57):

And people using Camunda 8 will recognize that this is one of those things, you've been building models, and adding transaction boundaries, to try and get around this limitation in the BPM specification about messaging, which says, "If you send a message, if there is nobody waiting for it, the message fails." That on a practical level doesn't work, because practically speaking, there's latencies, there's threads don't always catch up, and threads aren't a problem in Camunda 8. And the same with multi-instance parallel execution, this is something that there's all sorts of problems around optimistic locking. If you don't like the optimistic locking exception error that you've been getting in Camunda 7, Camunda 8 is going to make you very happy. And you can remove all of the weird hacky things you've done to try and solve the optimistic locking exceptions. Those who know, know what that means, those who don't, you are very lucky.

Bernd Ruecker (41:51):

No, 100%, I mean, these are one of the more detailed differences also in the architecture. Not using a relational database and transactions in a relational database, but things like a single write on an event lock, you don't have these contention problems. You can't have them. And this is really great. And I know exactly what you're were talking about. We had surprisingly many customers who did things like, hey, I want to kick off this one very often called batch process. Which then kicks off whatever, 4,000, 5,000, 6,000 of typed processes using the instance. And it not only costs optimistic locking exceptions, it costs a couple of different problems. And this is now super-great, we can solve that easily.

Niall Deehan (42:37):

And I think that's an important thing for people to realize when they're comparing Camunda 7 to Camunda 8, which is the fundamentals of Camunda 7 we've been dealing with for so long in both support cases and consulting missions, that we can now, when if we design something, think if only we had thought of this before, and then re-implement it in a way that means that we're building from a much more stable base database, that is thinking ahead to modern architectures. We never really built Camunda 7 for microservices at all, it was the opposite. It was intended to be embedded. And it grew into that microservice use case entirely down to the community.

Bernd Ruecker (43:24):

And this is part of what I said early on, we develop our applications differently than we did 10 years ago. And one part of that, is distributed systems, is remote calls. We are doing a rest call, a whatever call, a Kafka, messaging, something calling the mainframe. Something is always there, which is remote nowadays. It's unavoidable.

Niall Deehan (43:46):

Yeah, it is.

Bernd Ruecker (43:48):

And this is, I think one of the key architectural different primitives people are now thinking about, is like, we don't have asset transactions. Transactions with the database anymore in this environment. This would make its own podcast, Why? But I think the industry agreed on, we should assume it's not there. This means you have to learn new patterns around that. Some of them you are solved very well by a workflow engine, but it also means not having transaction integration with your workflow engine in my mind as a good thing, because it also means that you apply the same patterns as you do anyway, because you have your rest call on that level. This is one of the big areas where I think the whole industry still have to catch up a little bit with the speed of introducing microservices. Now we have to catch up learning distributed systems.

Niall Deehan (44:44):

Now we're coming to the end now, and I'm going to put you on the spot, and basically ask you, what thought would you like to leave Camunda 7 users with regards to Camunda 8? What is the underlying point you would like to have people be considering?

Bernd Ruecker (44:58):

I think you know me relatively well, so I'm this kind of, "It depends" person. I have problems pinpointing this to one thing, honestly. I think the whole change which is done there, technology architecture-wise makes a lot of sense to me. To what I see with the customers. I expect that actually to make a lot of things easier for our users. I really believe that. At the same time, I know that the existing Camunda 7 users, they love the product. Which I totally understand, I love the product as well. And we are with that product for a long time, so I'm also emotionally connected to that very deeply. I understand that change is also not super-easy. It's like, hey, we have these Java delegates, this Spring Boot app. It's so nice, why should we change that?

Bernd Ruecker (45:52):

And I totally understand that this takes a little bit of time to understand a couple of these contents. And I just want to motivate everybody who is still struggling with that. We have a couple of resources. For example, blog posts explaining certain things around, why not using Java delegates anymore? Why to use something like a remote engine and connect to it? How can I live without transactions? How can I do migrations? We're pulling together a lot of these material around the typical questions Camunda 7 users have. And we take that very seriously.

Bernd Ruecker (46:24):

I had the situation, I'm talking to a lot of customers at the moment, which are with us for a lot of years, like 5, 6, 7 years on the product, which always tell me like, "Hey, you said we should use the embedded engine and Spring Boot. Now what? And of course, I want to have good answers, and I think we have them. And if you're struggling with that, I totally understand that. And I would want just to motivate everybody to reach out, to find the right resources, to talk to us, because there are good reasons. And I'm pretty sure, we come to that conclusion. So far, all the customer interactions are super-positive.

Niall Deehan (47:01):

And if you have any problems with it, you know where to contact us. The forum is a good place to start. And from then on, there's a bunch of other random ways to get in touch with us. But the forum is a really nice one. Super, thank you very much Bernd. That was very enlightening. I'm looking forward to spending more time with Camunda 8 now, over the next year or so.

Bernd Ruecker (47:20):

Thanks for having me, Niall.

Niall Deehan (47:21):

We should definitely have another chat about this, once maybe we've seen some of the first big projects come about, so thanks again. Thanks everybody for listening, and I hope you enjoyed it. Goodbye.

Josh Wulf (47:38):

That’s a great conversation between Camunda Developer Advocate Niall Deehan and Camunda’s co-founder and chief technologist Bernd Ruecker. If you want to get your hands on Camunda Platform 8, go into some deep dive sessions, even do a four-hour Code Studio with Camunda Developer Advocates - a great opportunity to do that is at the upcoming Camunda Community Summit. It’s the online conference for Camunda developers, and it’s going to be really heavily focused on the new Camunda Platform 8. It’s April 26 - 28, 2022. To find out more about it and register go to: https://summit.camunda.com. 

Thanks for tuning in once again. Props to Inclusion Audio for the audio editing, and shoutouts to Shingo Nakamura and Monstercat.com for the soundtrack.