39. Challenge Accepted. with Vahid Gharavi, Director of Engineering at METRO.digital

"METRO AG is making billions having 751 shops in 25 countries. This is serious. What happens when COVID hits a business like that? How can IT and Digital Transformation projects help address the situation? What are Vahid's biggest challenges right now and how do you plan the architecture for systems handling hundreds of thousands of orders per day. It's B2B, it's integration, it's super scalable cloud operations. If you're a CTO, CDO, or CEO of a large-scale enterprise this episode is for you. If you're building products for enterprise software clients this episode is for you as well - and the insights how it works on a serious scale Vahid is sharing are just priceless!" Piotr Karwatka, Host.

Topics discussed:

What is the company, business model of METRO AG?

How did you join the company? What’s your role at METRO?

Tell us a little bit more about the role of eCommerce business at METRO?

How does Covid19 affect it?

What are the key challenges you and the company are facing right now in regards to the eCommerce business? 

How does it differ compared to the situation when you joined the company?

Let’s talk about the architecture that powers up 25 countries, bilions EUR rev. 

Cloud Native or Cloud Agnostic?

Which elements should you buy vs own?

What’s the optimal team structure at this scale?

What does the innovation process look like - from the new-feature req. up to deployment?

How do you integrate the features? How do you make it look and feel coherent? How do you optimize the experience?

How long does it take from the business decision for the feature to have it in production?

With still a significant amount of offline business how do you keep offline shops up-to date?

Which tech trends make you excited?


Transcript:


Piotr Karwatka: [00:00:39] METRO AG has a presence of 751 stores in 25 countries, one of the largest retailers in the world. Serving exclusively professionals such as restaurants, hotels, bars and cafes. Today I’m having Vahid Gharavi - director Of Engineering at METRO.digital. We’re going to talk about METRO's Digital Transformation including API First architecture and microservices. 

Hi, Vahid , thank you for accepting my invitation!

Vahid Gharavi: [00:01:08] Thank you. Good to be here. 

Piotr Karwatka: [00:01:09] Awesome. I guess we should start with the Metro AG. I mean, if you can say something about the company and the business model, it would be great. Not sure if everyone is on the same page with us. 

Vahid Gharavi: [00:01:20] Yeah, sure. So Metro we are a global wholesaler. Originally the company was founded in 1964, so I'm headquartered in Dusseldorf Germany. So we're one of those classical German organizations that have been around for a while and businesses the wholesale. So we are a wholesaler supplying mainly food but also non-food to in a B2B model.

So that means we supply quite a lot of Cora businesses and hurricanes are hotel restaurants, cafes but also small traders. Our customer segments are different from country to country. So in a lot of Western European countries, for example, our customers are mainly hurricanes. So we have a lot of restaurants and hotels and cafes, but in Eastern European countries the main customer group, our traders are small, small shops with so many super supermarkets we call it the trader business.

Piotr Karwatka: [00:02:13] It's originally B2B, but in some cases it works pretty much like B2C, I mean the cash and carry warehouses, right? Where the smaller retailers shops probably also coffee and restaurant owners?

Vahid Gharavi: [00:02:47] It's more or less. We have some cases where we have a, really a consumer as being able to inter our stores and, and shop. But in most of the countries, you need to have a Metro card.

So then you only are able to get a Metro card and get into our stores if you have a business registered. So if you have a registered business, if you have a legal business registered, then you're eligible to get a Metro card. And to shop with us, obviously, a lot of people who have or for example, freelancers or people have you know, one, one person businesses.

So not really large businesses, they also are entitled to get a Metro card. So a lot of these people are so probably are shopping for themselves that that does happen. But the main customer segment and most of our revenue is through the, through the real, real businesses. So that's really B2B. 

Piotr Karwatka: [00:03:33] I asked this question because it's led me to the, another one about the Skype, because I guess if you are supporting such a wide range of entities from, you know, one person companies up to a huge, you know, chains and hotels and so on, I guess the, the scale is, is really huge.

Can you give us some numbers, like to just get the scale. 

Vahid Gharavi: [00:03:58] Yeah, sure. So overall revenue of the group is around 26 billion. We have almost a hundred thousand employees throughout the world with the 25 countries where we are in. Yeah, so that definitely one of the main main wholesalers on the globe and to online tracks, I mean, we still obviously have a huge offline presence.

But we've had a shift to also online orders and online orders. We have roughly 3.2 billion orders per year. And this is excluding our China business because we had a PR we have a presence. We had a presence in China, but this business is now for the majority salt therefore removing the sales figures of what used to be our China business.

So it's without, without that it is currently around 3.2 billion. That's just on the online orders. That's really a lot. 

Piotr Karwatka: [00:04:51] So person wise is like how we should measure it. Like it's double digit or the revenue from the, the percentage of the revenue from online versus offline. So like 20%. And do I count it correctly? 

Vahid Gharavi: [00:05:11] It's exactly. It's around 10 to 15, 15%, 15 to 20%. So that would be a percentage and growing, obviously, I mean, you should, you should imagine that some, some years ago this was zero because we were purely like cash and carry. So, the whole business model over time has been that you come to our stores and you buy your stuff and go, we're not delivering to you.

But the delivery business started some years ago. And that's me, you know, growing extremely fast, as you can imagine, 

Piotr Karwatka: [00:05:42] it's a pretty good number and, but still a lot of space for growth there. And that does not really lead us to the next question, like how the COVID-19 affected this revenue from online, I guess it affected out.

Well, I can imagine the fact that a lot positively, but also negatively because of the lockdowns. 

Vahid Gharavi: [00:06:03] Yes. Yeah. So in the countries where we have a big you know, hospitality customer, customer base, obviously this was a, this was affected because when the restaurants are closed for a longer periods of time, this does obviously affect our business in a negative way.

In other areas. There were some actual improvements. So for example, in those countries where we have the trader a trader business, that means it's not restaurants, but it's small supermarkets. Oh, Grocery shops. Exactly. So we actually had some customer segments that showed the big plus but other customer segments showed the negative.

Overall. Obviously this has affected our business in a negative way. But it's been at a scale that we've been able to stop so far. 

Piotr Karwatka: [00:06:45] Gotcha. That's interesting because Yeah. That's not that, not that obvious right? In this segment on the one hand, everyone I invited for interviews said something like, yeah, COVID-19 sped up the whole digital transformation a lot.

And everything is growing. But this is true if you are not serving those affected segments. Right. And, and here is the more complicated but good to hear that some, some segments we're growing anyway. Okay. How did you join the company and what's your role at Metro?

Vahid Gharavi: [00:07:24]l I joined the company in 2015, 2016. So February, 2016. So in a couple of years, I'll be five years at the company and yeah, so I joined the company actually in my previous job, I was working as a CTO. I'm a co-founder and CTO at a startup Where I was for about four years. And while working for the startup, we also had to add some interactions with Metro, because Metro used to be one of the customers that we worked with. And as I decided to look for a new challenge I realized that there was a huge digital transformation project going on and having been started at Metro.

And that was actually for me the motivation to be, to be part of that obviously this such a digital transformation at Metro, as you can imagine by the numbers, I told you a it's something pretty, pretty big. And that was exactly what had a lot of impacts. So I joined Metro and worked on the, on the web shop on a, on a new web shop that we were developing in 2015 to 2016.

And this was a new self-made webshop built with microservices on a microservice. Sorry. Texture. And so when I joined in February on the, in the April of the same year, shortly after this was launched this was launched. And this is the one where we are currently running more than 3 billion years of sales a year with.

And so I have started working on the web shop. And so working on the, on the product side, so we're trying to shape a shape, shape this, a workshop product to, to our business. And after around two and a half years, halfway through, I switched to the engineering organization.

So my current role is Director of Engineering. So I am responsible for the engineering units. And all of the software engineers that we have in the company. So we have a matrix organization where we have different reporting lines for engineers or UX professionals. And they all, obviously at the end work in the product teams.

So my job is to oversee this engineering organization, hiring engineering culture and technical architecture, and all of those things. 

Piotr Karwatka: [00:09:28] Gotcha. Pretty interesting. So it's, it's changed. It has changed over the years from a pretty well defined single project, like implementing this new issue, to a more broad role of overseeing all the engineering initiatives of the company.

That's, that's, that's probably an interesting role because I can imagine that each day you have different challenges to face, right? It's not like being single. 

Vahid Gharavi: [00:09:56] Yeah, exactly. So in my previous job, I was working on the web shop products. And that was my main focus, but currently you know having, I have an interface with multiple different products that we, that we have at the company, because all of them have, have engineers working with them.

So that's in that sense there's more variety and that's for sure. Gotcha. 

Piotr Karwatka: [00:10:15] That's super interesting. So let me ask you about the Architecture that pours up those 25 countries. Metro is operating in billions of Euro revenue how the architecture looks at it. 

Vahid Gharavi: [00:10:28] Well, it's really a heterogeneous architecture because we have quite a lot of software that has been self-built over the years.

So we've been in business for quite a long time and we've had tons of legacy running our business, which is extremely important to our business. It's the backbone of our business, but we also have a lot of new applications that are like really fully cloud native applications. That we are, that were built over the last couple of years.

And so we've got a really heterogeneous picture. So I cannot give you like it's not like one architecture that would fit everywhere. But what I could say is that in most of the cases, the newer applications that we're building. Ourselves the focuses that we built, the applications ourselves that are differentiating for our business and there were differentiating applications as are most of the time built on a, on a microservice architecture, but not always.

So in most of the cases, yes, Because when we started with the digital transformation, obviously scalability was one of the important requirements for the software that we built because it has to be able to, you know, cover the whole range of our business in all the countries and and the number of customers that we have and the revenue that we have.

So in that sense microservices was one of the main to grow architectures for the newer applications. 

Piotr Karwatka: [00:11:45] Gotcha. That makes perfect sense. So it seems like a kind of transformation of the attitudes towards building the software. I mean, you started with your own everything approach where you build everything from scratch.

I think it's like. Pretty natural for many other businesses as well. When I spoke to other folks, it's very often the case. We, like, we start with our own ERP because there was nothing like this on the market and so on, but it's changed over time. And you already partially answered my question, which was which elements to build versus, oh, you said that to build those that make the difference for the customer and for the business.

It makes perfect sense. When we were preparing for this interview, you also told me that you actually also have the second webshop, but based on the platform on Spriker right. If you compare the pros and cons between maintaining this 100% own webshop. And maybe new apps will do microservices, but still built by your team versus having a platform as a base and are built by you.

How do you compare it? Like what's your feeling? Because I guess there is nothing to ask for. Right? Probably there are cases. One is better than another, but can you wrap it up somehow? 

Vahid Gharavi: [00:13:15] Yeah, that's a specific example that, you know, put your finger on. It's obviously tricky, tricky to give a, to give a straight answer to this one.

In general. So the webshop board that we started in with 2015 was completely self-built, completely self-worth based on microservices and we chose an architecture. We chose an organizational setup. Where we split the shop into different verticals. So we said you know, we split the shop into the, like the search and the discovery features where all one, one particle alibi, one product team.

And for example, the basket and all the, all the things around the basket for capturing an order they were belonging to another team. So we, we, we split it into these verticals. So it was also organizationally scalable for us. So, the way that we set it up, Completely handed me to what customer microservice has enabled us to really make that separation between the teams.

So it was like an organizational benefit that we had and it gave us freedom to shape the shop. Exactly, you know, as, as we, as we wanted it to enable us to have those teams separate in that way. And looking back if you look at the success of the shop, more than 3.2 billion euros of revenue per year, I think that the investment that's been a success and that has been a good investment.

And then we used for the trader shop. And in the beginning, it was also for us a bit of an experimentation because we wanted to see how different model. Architectural model in terms of buying, buying, buying the shop functionality, the base headless functionality and building the shop as we wanted with with our own for our own business on top, how would this would fare.

And this has also been quite a success because the trader shop based on Spriker is is growing extremely fast has been skating, pretty good. It gave us quite a lot of features out of the box. So things like search functionality or being able to browse through your products, product, database and a bunch of things like that, which just come just in there, you know, out of the box.

Is, is also a big advantage, but the organizational setup that we have with Spriker is is different. We have obviously much smaller footprints when it comes to revenue and orders compared to our main, main workshop. This, this could grow and change but currently make, it would make a comparison a little bit more difficult.

And we also have one one team working on this. So we don't have that vertical split which might be also kind of related to the scale of the shop. Maybe if the shop would grow much bigger than it is now, if the orders would be much bigger, if the countries would be much more and products would be assortment would be much, much bigger.

Maybe we would justify also splitting that up into multiple different teams and maybe. Then architecture might be something we need to discuss. How does the architecture allow us to split into multiple different teams? So I can not give a straight answer, but only maybe some of the learnings and observations through the years.

But in general, like I said we wanna, we want to build the stuff only, which makes a difference for our business because there is a, in our business, I mean, we have a lot of software, which is running our stores and a lot of software, which is running all of our online solutions obviously. And the question we try to ask is, do we, does this really make a difference for our business, that we have to really build ourselves or not?

In the beginning, we answered yes. Quite a lot. So we started building quite a lot of things, but now we're trying to be more selective and really focus on the things that make a difference for our business and try to buy the others if they are, if they could be standard solutions, for example.

Absolutely. 

Piotr Karwatka: [00:16:43] Totally get your point. So let me make the switch to a more general question about the microservices and your experience with this architecture. You already said that The way you structure it is by the products and having different products, teams working on uncertain differences on the checkout product page and so on.

Pretty common architecture for microservices seems like work very well. I had a great talk with the software engineering manager at Zalando, and they have pretty much the same architecture but he also pointed out the downsides. Especially how the complexity that comes with these structures, what are your feelings?

Vahid Gharavi: [00:17:21] The downside is obviously that you will have a lot more alignment to do. So your teams have to spend a lot of time talking to each other. You will not manage to have teams completely separated from each other that they don't have to talk to each other and they just can build their stuff by themselves.

No, obviously they will need each other. They have dependencies, this will become a bit of a headache. This will slow things down. So obviously your overall development will slow down because of such alignments being necessary. 

Piotr Karwatka: [00:17:52] And sorry, how have you aligned them? Is it like a kind of process where you have meetings or maybe you are based on, on the roadmaps and change logs?

Vahid Gharavi: [00:18:02] So this is for the, for the OKR cycle. In the beginning, we have objectives for the shop. And then we have objectives on, on the team level that are contributing to the overall shop objectives. And when the teams come up with their objectives, they have to make sure that if they need something from another, another team, For it to deliver their objectives, they need to talk at that point right at the beginning and say, guys, are you going to support us to reach this objective?

Because that means you have to give us, you know, 40% of your time. And if the answer is yes, then that alignment is made there. And then in the beginning we also have several practices. We have a for example six we have six sprint breakdown that we do in the, in the beginning of the, of the OKR cycle.

And during this the teams are really trying to map out, for example, at which sprints can they expect something from another team so they can accordingly. Plan building something because they, for example, if they get the dependency delivered in sprint two than they could in sprint three, start building something which should realize that.

So this is, in general, how we do alignments, but I have to say dependency management and alignments at such a company like ours is, will remain a pain. It's always a pain. And this is one of the main issues that slows us down because the dependencies are just quite a, really a lot. And it creeps through the entire organization.

Piotr Karwatka: [00:19:25] So let's see how I actually imagined it in the works. So maybe, can you give us some examples of the most complex, horizontal feature being implemented across all of those things? I mean, the situation when you have such a feature, like for example, Probably changing up the business model for some category of products or something like this, and that needed to be changed on insert checkout product page everywhere.