About this video
A practical walkthrough of technical AEO/GEO/AISEO checks you can run on any website. Covers disabling JavaScript to see what crawlers see, pagination without JS, markdown content negotiation for A…
Full transcript
Edited transcript from the original Roboto page
AEO, GEO, AISEO, they're all basically the same thing despite what people argue about the nomenclature. What matters is that they work. We've been monitoring them internally and you can apply them on your own website, or you can talk to a Next.js Sanity agency if you want help building this yourself.
Let's start with the website itself. This is something you can do without being a developer and still get the same result. If you right-click on any website and hit inspect, there are two things I'd advise doing.
First, it might not look like this straight away. It probably looks cluttered. On a Mac, press command, shift, and P. On Windows it's control, shift, and P. Type disable JavaScript. You may notice immediately nothing's really changed. But if you hit command and R to refresh the page, you'll notice immediately that certain elements have gone. The reason is they're JavaScript. This is roughly what Google sees, and it's a great way of navigating your own website to see what's going on.
There are some caveats. If you look at the left-hand side you'll see exactly what's going on. For example, if I press here you're going to see things like divs, you're going to see things like P tags, which basically just means paragraph. This is how you can actually see the content on your page and work out exactly what's happening. JavaScript's turned off, so I can see all of this stuff on the page. From an SEO perspective, you're doing something very good. From GEO and AEO perspective, you're also doing something very good, but not quite the best, which we'll talk about next.
This is especially important. I'll show you where everybody screws it up so you can not screw it up yourself. Let's enable the JavaScript again because my navigation won't work, and let's go over to blog. You can make your navigation work without JavaScript. We purposely chose not to do that. There is a way around it and you can do clever stuff, but for the sake of this conversation, keep it simple.
When you have pages like this, I'll show you what everybody screws up. Hit command shift P again and disable JavaScript. Refresh the page. Lines have gone. We see where we're going with this, but the blog posts are still here. More importantly, we have pagination, which means if a crawler hits the website they need to go through the pagination. They need to see more than just the first set of articles. Let's go to page two. Great, I can see page two. Let's go to page three. This is really important. Same principle applies. Categories don't work right now, so that's something I'm going to fix after this video, but you can see the point. The more you can do without JavaScript, the better the website generally indexes. This is a general rule of thumb, so don't hold it as the be-all and end-all, but it is very important. It also means that a lot of this stuff can get cached if you're rendering it statically. There are nuances with rendering techniques, but for the sake of doing one big overall view, this is the first thing we do when assessing a website for SEO and AEO.
One thing I actually stole from a Profound presentation is a website called RunRepeat, which is quite useful. RunRepeat is very interesting because it gets cited a lot. The reason it gets cited so much is that everything you're seeing on the page here is unique, interesting, numerical information that AI can't find literally anywhere else. If you have unique, numerical, interesting information and most importantly AI can't find it anywhere else, you're going to do pretty well on citations. If you look up something like how much pressure can a Nike Vomero take on my feet, or how long will they last, you'll get this information. This is very important.
The other thing I've left off so far is content negotiation. Content negotiation sounds scary, but it's not actually that scary. What this basically does is give a way for AI crawlers to see your website. This isn't a strict rule before I say this, but most AI crawlers do not run JavaScript on the website, which is why I was showing you that for the first part. But the interesting thing here is when AI hits a website, it looks like this. This is what an AI will basically see. This looks like the Matrix. Nobody can read this. It's really hard to see what is here. However, let's try this again but with this nice little bit at the end, which is accept text markdown. You have to actually do this on your website. This isn't just something you can say "go throw this at the end, it's done for you." It actually takes a lot of work, and it's very important that you have parity between the HTML version and the markdown version.
What we're going to do is copy this exact thing over again and paste it back in. This is Vercel's, not our own, but I can also put our own on here and it will show you exactly the same thing. Is it going to look like the Matrix part two? No. It's actually going to look like markdown, which is really important. If you think about it, AI crawlers and how we use AI in general is we try not to blow the context window. If you want to know how that works, it basically means the more information you put into AI, generally the worse it gets. There's a sweet spot around 20% of the actual context window depending on every model. Instead of having all this crap where it bloats that context window when things are crawling, you get this where it tells you exactly what you need to know. More importantly, Vercel did a little teardown of how this actually works. The most interesting thing was that it was 3 KB, so a 99.37% reduction in payload size instead of 500 KB. This is super important.
What I will say is the pitfall most people fall into if they're doing this for their own website is the fact that there isn't parity. If you get a 1 KB payload but it only has half the information of the page, that's far worse than just using the original HTML. What you've got to make sure you do is that the markdown version is in parity with the HTML version. In some cases you need to expand things. For example, an FAQ section, if it's inside a component, then you need to make sure that FAQ actually gets brought into markdown in a way that's nice. There are loads of intricacies depending on what you're doing here. The best way to do this is just give us a shout and we can talk you through it.
Let me show you a couple more things. This is Tetris from Wikipedia. I'm sure you've seen this before. However, something you probably never thought about is why did Wikipedia end up doing so well in the first place, and why does it appear so high on Google? One of the many reasons is if you look at the left-hand side, this is a table of contents. If you look, we have an unordered list and then list items. You can see table of content, list item, list item, list item. It's all semantically correct, which basically just means it's the way HTML should be written. It's the way computers can easily interpret what's actually going on on the screen. There's not a tremendous amount of JavaScript being fired here. Whenever we're building something like a blog, you're going to make sure you have a table of contents similar to Wikipedia. You don't necessarily want to have all the JavaScript, or it should still run and be semantically correct without JavaScript. All of these different little bits and pieces add up.
Last one I want to show you as well, we're going to right-click and hit inspect. You don't need a developer involved to do this. Just click into this thing on the left. Doesn't matter where you click. Try not to click a link. Just try to click anywhere in here and press command F or control F if you're on Windows and type JSON. Now what you're looking for is JSON-LD on the page. Let me have a look. This is what I'm looking for. You see breadcrumb JSON-LD. Pop this open and you can see the information about the breadcrumb inside here. Let me look for another one. Okay, collection page JSON-LD for blog. Great. This is showing me what's actually on the page from a JSON-LD perspective. I understand what's going on here and by proxy, a computer understands it too. This is really important to essentially show that you know what's going on on the page and you're presenting that to Google and all these different AI crawlers. It's really helpful to give some rich structured data.
Final thing worth doing here is there is a JSON validator from Google. I won't show it in this session in particular, but you should always just check at one level that the JSON is valid. I know there's a couple in here where we need to do a couple of bits of tweaking to make sure the JSON is valid, but it's always important to do that.
This gives you a rough idea of everything you can get started with. This is just scratching the surface of what AEO, SEO, and all of these different functionalities are about. What I want to do is have more deep dives and talk more about it in the future if you're particularly interested.
Read timestamped source captions
0:04Hey folks, how's it going? Hope you're
0:04Hey folks, how's it going? Hope you're having a great day today. So, I want to
0:05having a great day today. So, I want to
0:05having a great day today. So, I want to talk to you about AEO,
0:08talk to you about AEO,
0:08talk to you about AEO, GEO,
0:09GEO,
0:09GEO, AISEO.
0:11AISEO.
0:11AISEO. Okay? They're all basically the same
0:13Okay? They're all basically the same
0:13Okay? They're all basically the same thing, despite what everybody's going to
0:14thing, despite what everybody's going to
0:14thing, despite what everybody's going to say and argue about the nomenclature,
0:17say and argue about the nomenclature,
0:17say and argue about the nomenclature, but trust me on this when I tell you
0:18but trust me on this when I tell you
0:18but trust me on this when I tell you today we're going to talk about the
0:19today we're going to talk about the
0:19today we're going to talk about the technical. These things work. We've been
0:21technical. These things work. We've been
0:21technical. These things work. We've been monitoring them internally ourselves,
0:23monitoring them internally ourselves,
0:23monitoring them internally ourselves, and you can apply them on your own
0:24and you can apply them on your own
0:24and you can apply them on your own website, or you can go and talk to the
0:27website, or you can go and talk to the
0:27website, or you can go and talk to the best Next.js sanity agency around
0:32best Next.js sanity agency around
0:32best Next.js sanity agency around uh if you want to help in building this
0:34uh if you want to help in building this
0:34uh if you want to help in building this yourself. So, anyway, let's go through
0:36yourself. So, anyway, let's go through
0:36yourself. So, anyway, let's go through this.
0:37this.
0:37this. Um let's start with the actual website
0:39Um let's start with the actual website
0:39Um let's start with the actual website itself. Now, this is something you can
0:41itself. Now, this is something you can
0:41itself. Now, this is something you can do that you're not necessarily a
0:42do that you're not necessarily a
0:42do that you're not necessarily a developer, but you can still get the
0:43developer, but you can still get the
0:43developer, but you can still get the same end result. If you right click on
0:45same end result. If you right click on
0:45same end result. If you right click on any website and you hit inspect,
0:48any website and you hit inspect,
0:48any website and you hit inspect, there's two things I would advise doing.
0:50there's two things I would advise doing.
0:50there's two things I would advise doing. So, first of all, it might not look like
0:52So, first of all, it might not look like
0:52So, first of all, it might not look like this straight away. It probably looks
0:53this straight away. It probably looks
0:53this straight away. It probably looks something like this. Now, that if you're
0:55something like this. Now, that if you're
0:55something like this. Now, that if you're on a Windows machine, sorry, sorry in
0:56on a Windows machine, sorry, sorry in
0:56on a Windows machine, sorry, sorry in advance. Um
0:58advance. Um
0:58advance. Um if you're on a Mac, what you can do is
1:00if you're on a Mac, what you can do is
1:00if you're on a Mac, what you can do is you can press command, shift, and P. I
1:01you can press command, shift, and P. I
1:01you can press command, shift, and P. I think on Windows it's control, shift,
1:03think on Windows it's control, shift,
1:03think on Windows it's control, shift, and P. And you can type disable
1:05and P. And you can type disable
1:05and P. And you can type disable JavaScript. Um now, you may notice
1:08JavaScript. Um now, you may notice
1:08JavaScript. Um now, you may notice immediately nothing's really changed.
1:10immediately nothing's really changed.
1:10immediately nothing's really changed. However, if I hit command and R, we
1:13However, if I hit command and R, we
1:13However, if I hit command and R, we refresh the page. Now, you'll notice
1:14refresh the page. Now, you'll notice
1:15refresh the page. Now, you'll notice immediately that my lines have gone on
1:16immediately that my lines have gone on
1:16immediately that my lines have gone on the front page. And the reason these
1:18the front page. And the reason these
1:18the front page. And the reason these lines have gone is because they are
1:19lines have gone is because they are
1:19lines have gone is because they are JavaScript. So, this is roughly what
1:22JavaScript. So, this is roughly what
1:22JavaScript. So, this is roughly what Google sees, and this is a great way of
1:24Google sees, and this is a great way of
1:24Google sees, and this is a great way of navigating your own website
1:26navigating your own website
1:26navigating your own website to see what's going off. Now, there are
1:29to see what's going off. Now, there are
1:29to see what's going off. Now, there are some caveats. So, if you have uh HTML on
1:33some caveats. So, if you have uh HTML on
1:33some caveats. So, if you have uh HTML on the page, so if you're looking at the
1:34the page, so if you're looking at the
1:34the page, so if you're looking at the left-hand side and it shows you exactly
1:36left-hand side and it shows you exactly
1:36left-hand side and it shows you exactly what's going on. So, for example, if I
1:38what's going on. So, for example, if I
1:38what's going on. So, for example, if I press this little thing up here, and I
1:40press this little thing up here, and I
1:40press this little thing up here, and I press here, you're going to see things
1:41press here, you're going to see things
1:41press here, you're going to see things like divs, you're going to see things
1:43like divs, you're going to see things
1:43like divs, you're going to see things like P tags, which basically just means
1:45like P tags, which basically just means
1:45like P tags, which basically just means paragraph. Um this is how you can
1:46paragraph. Um this is how you can
1:46paragraph. Um this is how you can actually see the content on your page,
1:48actually see the content on your page,
1:48actually see the content on your page, and you can go through the page, and you
1:49and you can go through the page, and you
1:49and you can go through the page, and you can work out exactly what's going on.
1:52can work out exactly what's going on.
1:52can work out exactly what's going on. So, again, JavaScript's turned off, I
1:54So, again, JavaScript's turned off, I
1:54So, again, JavaScript's turned off, I can see all of this stuff on the page.
1:56can see all of this stuff on the page.
1:56can see all of this stuff on the page. Now, the beauty of this is that from an
1:58Now, the beauty of this is that from an
1:58Now, the beauty of this is that from an SEO perspective, you're doing something
2:00SEO perspective, you're doing something
2:00SEO perspective, you're doing something very, very good. And from GEO from AEO
2:03very, very good. And from GEO from AEO
2:03very, very good. And from GEO from AEO perspective, you're also doing something
2:05perspective, you're also doing something
2:05perspective, you're also doing something very, very good, but not quite the best,
2:07very, very good, but not quite the best,
2:07very, very good, but not quite the best, which we're going to talk about in a
2:08which we're going to talk about in a
2:08which we're going to talk about in a second. So, for now,
2:11second. So, for now,
2:11second. So, for now, um this is quite important and this is
2:12um this is quite important and this is
2:12um this is quite important and this is especially important. I'll tell you
2:13especially important. I'll tell you
2:13especially important. I'll tell you where everybody screws it up so you can
2:15where everybody screws it up so you can
2:15where everybody screws it up so you can not screw it up yourself. Let's enable
2:17not screw it up yourself. Let's enable
2:17not screw it up yourself. Let's enable the JavaScript again
2:19the JavaScript again
2:19the JavaScript again um because my navigation won't work and
2:20um because my navigation won't work and
2:20um because my navigation won't work and let's go over to blog.
2:23let's go over to blog.
2:23let's go over to blog. Now, you can make your navigation work
2:25Now, you can make your navigation work
2:25Now, you can make your navigation work without JavaScript. We've purposely
2:27without JavaScript. We've purposely
2:27without JavaScript. We've purposely chose not to do that. There is a way
2:29chose not to do that. There is a way
2:29chose not to do that. There is a way around it and you can do some clever
2:31around it and you can do some clever
2:31around it and you can do some clever stuff. Uh but for the sake of this
2:32stuff. Uh but for the sake of this
2:32stuff. Uh but for the sake of this conversation, just keeping it nice and
2:34conversation, just keeping it nice and
2:34conversation, just keeping it nice and simple. Uh
2:35simple. Uh
2:35simple. Uh when you have pages like this, I'll show
2:37when you have pages like this, I'll show
2:37when you have pages like this, I'll show you what everybody screws screws up. So,
2:39you what everybody screws screws up. So,
2:39you what everybody screws screws up. So, if you hit command shift P again and
2:40if you hit command shift P again and
2:40if you hit command shift P again and let's disable JavaScript, right? Let's
2:42let's disable JavaScript, right? Let's
2:42let's disable JavaScript, right? Let's refresh the page. Okay, lines have gone.
2:44refresh the page. Okay, lines have gone.
2:44refresh the page. Okay, lines have gone. We see where we're going with this, but
2:46We see where we're going with this, but
2:46We see where we're going with this, but the page, the blog posts are still here,
2:50the page, the blog posts are still here,
2:50the page, the blog posts are still here, okay? Now, more importantly, we have
2:52okay? Now, more importantly, we have
2:52okay? Now, more importantly, we have pagination, which means that if a
2:53pagination, which means that if a
2:54pagination, which means that if a crawler hits the website, they need to
2:55crawler hits the website, they need to
2:55crawler hits the website, they need to go through the pagination. They need to
2:57go through the pagination. They need to
2:57go through the pagination. They need to see more than just I think it's 12 or 9
2:59see more than just I think it's 12 or 9
2:59see more than just I think it's 12 or 9 or something, more than the first set of
3:01or something, more than the first set of
3:01or something, more than the first set of articles. So, let's go to the next one.
3:03articles. So, let's go to the next one.
3:03articles. So, let's go to the next one. Let's go to page two.
3:05Let's go to page two.
3:05Let's go to page two. Great, I can see page two.
3:07Great, I can see page two.
3:07Great, I can see page two. Let's go to page three.
3:12You see what I'm saying? This is really
3:12You see what I'm saying? This is really important. Same principle, I think these
3:14important. Same principle, I think these
3:14important. Same principle, I think these still work. I want to see the
3:15still work. I want to see the
3:15still work. I want to see the categories. Okay, categories don't work
3:17categories. Okay, categories don't work
3:18categories. Okay, categories don't work right now. So, that's something I'm
3:18right now. So, that's something I'm
3:18right now. So, that's something I'm going to go fix after this video, but
3:20going to go fix after this video, but
3:20going to go fix after this video, but you can see the point of what I'm
3:21you can see the point of what I'm
3:21you can see the point of what I'm showing you, which is that the more you
3:24showing you, which is that the more you
3:24showing you, which is that the more you can do without JavaScript, the better
3:25can do without JavaScript, the better
3:26can do without JavaScript, the better the website generally indexes. This is a
3:28the website generally indexes. This is a
3:28the website generally indexes. This is a general rule of thumb, so don't hold
3:30general rule of thumb, so don't hold
3:30general rule of thumb, so don't hold this as like uh you know, like as the
3:34this as like uh you know, like as the
3:34this as like uh you know, like as the the be-all and end-all, but it is very,
3:35the be-all and end-all, but it is very,
3:35the be-all and end-all, but it is very, very important. It also means that a lot
3:38very important. It also means that a lot
3:38very important. It also means that a lot of this stuff can get cached if you're
3:40of this stuff can get cached if you're
3:40of this stuff can get cached if you're rendering it statically. Uh again, there
3:43rendering it statically. Uh again, there
3:43rendering it statically. Uh again, there are some nuances with rendering
3:44are some nuances with rendering
3:44are some nuances with rendering techniques and things, but just for the
3:45techniques and things, but just for the
3:45techniques and things, but just for the sake of doing one big overall what you
3:48sake of doing one big overall what you
3:48sake of doing one big overall what you should be looking for, this is the first
3:50should be looking for, this is the first
3:50should be looking for, this is the first thing that we do when we look at
3:52thing that we do when we look at
3:52thing that we do when we look at actually assessing a website as far as
3:54actually assessing a website as far as
3:54actually assessing a website as far as SEO, AEO, GEO, AISEO. From now on, I'm
3:57SEO, AEO, GEO, AISEO. From now on, I'm
3:57SEO, AEO, GEO, AISEO. From now on, I'm just going to say SEO and AEO because
4:00just going to say SEO and AEO because
4:00just going to say SEO and AEO because it's the ones I'm used to.
4:02it's the ones I'm used to.
4:02it's the ones I'm used to. Now, um one thing that I actually stole
4:04Now, um one thing that I actually stole
4:04Now, um one thing that I actually stole from one of Profound's uh presentation
4:07from one of Profound's uh presentation
4:07from one of Profound's uh presentation is this website, which is quite useful.
4:09is this website, which is quite useful.
4:09is this website, which is quite useful. So, this website is RunRepeat.
4:12So, this website is RunRepeat.
4:12So, this website is RunRepeat. And why is RunRepeat so interesting? In
4:14And why is RunRepeat so interesting? In
4:14And why is RunRepeat so interesting? In fact, let me pop up the next tab.
4:16fact, let me pop up the next tab.
4:16fact, let me pop up the next tab. RunRepeat is very, very interesting
4:17RunRepeat is very, very interesting
4:17RunRepeat is very, very interesting because it gets cited a lot, like a lot,
4:21because it gets cited a lot, like a lot,
4:21because it gets cited a lot, like a lot, a lot.
4:22a lot.
4:22a lot. Um and the reason why it gets cited so
4:24Um and the reason why it gets cited so
4:24Um and the reason why it gets cited so much, you can probably tell immediately,
4:26much, you can probably tell immediately,
4:26much, you can probably tell immediately, is that everything that you're seeing on
4:28is that everything that you're seeing on
4:28is that everything that you're seeing on the page here is unique, interesting,
4:31the page here is unique, interesting,
4:31the page here is unique, interesting, numerical information that an AI can't
4:35numerical information that an AI can't
4:35numerical information that an AI can't find literally anywhere else. So, if you
4:37find literally anywhere else. So, if you
4:37find literally anywhere else. So, if you have unique, numerical,
4:41have unique, numerical,
4:41have unique, numerical, interesting information, and most
4:43interesting information, and most
4:43interesting information, and most importantly, AI can't find it anywhere
4:46importantly, AI can't find it anywhere
4:46importantly, AI can't find it anywhere else,
4:48else,
4:48else, you're going to do pretty well on the
4:49you're going to do pretty well on the
4:49you're going to do pretty well on the citations. So, if you go look up some of
4:51citations. So, if you go look up some of
4:51citations. So, if you go look up some of these, you know, you're like, "Oh, how
4:53these, you know, you're like, "Oh, how
4:53these, you know, you're like, "Oh, how you know, how much pressure can a Nike
4:54you know, how much pressure can a Nike
4:55you know, how much pressure can a Nike Vomero take on my feet?" This is going
4:57Vomero take on my feet?" This is going
4:57Vomero take on my feet?" This is going to tell you. How long will they last,
4:59to tell you. How long will they last,
4:59to tell you. How long will they last, right? You will get this information.
5:01right? You will get this information.
5:01right? You will get this information. So, this is very, very important. Now,
5:03So, this is very, very important. Now,
5:03So, this is very, very important. Now, the other thing that I've left off so
5:05the other thing that I've left off so
5:05the other thing that I've left off so far is content negotiation. And content
5:09far is content negotiation. And content
5:09far is content negotiation. And content negotiation sounds real scary, but it's
5:11negotiation sounds real scary, but it's
5:11negotiation sounds real scary, but it's not actually all that scary. Now,
5:14not actually all that scary. Now,
5:14not actually all that scary. Now, what this basically does is it gives a
5:16what this basically does is it gives a
5:16what this basically does is it gives a way for AI crawlers to be able to see
5:19way for AI crawlers to be able to see
5:19way for AI crawlers to be able to see your website.
5:20your website.
5:20your website. This is not a strict rule before I say
5:23This is not a strict rule before I say
5:23This is not a strict rule before I say this, but most AI crawlers do not run
5:26this, but most AI crawlers do not run
5:26this, but most AI crawlers do not run JavaScript on the website, which is the
5:27JavaScript on the website, which is the
5:27JavaScript on the website, which is the reason why I was showing you that for
5:29reason why I was showing you that for
5:29reason why I was showing you that for the first part. But the interesting
5:31the first part. But the interesting
5:31the first part. But the interesting thing here
5:32thing here
5:32thing here is when AI hits a website, it looks
5:35is when AI hits a website, it looks
5:35is when AI hits a website, it looks something
5:36something
5:36something like this. So, let me take off this bit.
5:40like this. So, let me take off this bit.
5:40like this. So, let me take off this bit. This is what a AI will basically see.
5:44This is what a AI will basically see.
5:44This is what a AI will basically see. You see this? This looks like the
5:46You see this? This looks like the
5:46You see this? This looks like the Matrix. People don't read this. I can't
5:48Matrix. People don't read this. I can't
5:48Matrix. People don't read this. I can't read this. Nobody can read this. It's
5:50read this. Nobody can read this. It's
5:50read this. Nobody can read this. It's really hard to see what is it here.
5:53really hard to see what is it here.
5:53really hard to see what is it here. However,
5:54However,
5:54However, let's try this again, but we have this
5:56let's try this again, but we have this
5:56let's try this again, but we have this nice little bit at the end, which is
5:58nice little bit at the end, which is
5:58nice little bit at the end, which is accept text markdown. Now, you have to
6:00accept text markdown. Now, you have to
6:00accept text markdown. Now, you have to actually do this on your website. This
6:01actually do this on your website. This
6:01actually do this on your website. This isn't just something that I can say,
6:03isn't just something that I can say,
6:03isn't just something that I can say, "Oh, yeah, go throw this at the end.
6:05"Oh, yeah, go throw this at the end.
6:05"Oh, yeah, go throw this at the end. It's done for you." It actually takes a
6:07It's done for you." It actually takes a
6:07It's done for you." It actually takes a lot of work, and it's very important
6:09lot of work, and it's very important
6:09lot of work, and it's very important that you actually have parity between
6:10that you actually have parity between
6:10that you actually have parity between the HTML version and the markdown
6:13the HTML version and the markdown
6:13the HTML version and the markdown version. So, what we're going to do is
6:14version. So, what we're going to do is
6:14version. So, what we're going to do is we're going to copy this exact thing
6:16we're going to copy this exact thing
6:16we're going to copy this exact thing over again.
6:17over again.
6:17over again. And we're going to go to the bottom
6:18And we're going to go to the bottom
6:18And we're going to go to the bottom here, and we're going to paste it back
6:19here, and we're going to paste it back
6:19here, and we're going to paste it back in. Okay? This is Vercel's. This is not
6:21in. Okay? This is Vercel's. This is not
6:21in. Okay? This is Vercel's. This is not our own, but I can also put our own on
6:23our own, but I can also put our own on
6:23our own, but I can also put our own on here, and it will show you exactly the
6:24here, and it will show you exactly the
6:24here, and it will show you exactly the same thing. Let's see what we get. Is it
6:27same thing. Let's see what we get. Is it
6:27same thing. Let's see what we get. Is it going to look like the Matrix part two?
6:29going to look like the Matrix part two?
6:29going to look like the Matrix part two? No. It's actually going to look like
6:32No. It's actually going to look like
6:32No. It's actually going to look like markdown, which is really important
6:33markdown, which is really important
6:33markdown, which is really important because if you think about it, AI
6:35because if you think about it, AI
6:35because if you think about it, AI crawlers and how we use AI in general is
6:38crawlers and how we use AI in general is
6:38crawlers and how we use AI in general is we try not to blow the context window,
6:40we try not to blow the context window,
6:40we try not to blow the context window, which if you want to know how that
6:41which if you want to know how that
6:41which if you want to know how that works, it basically means the more
6:43works, it basically means the more
6:43works, it basically means the more information you put into AI,
6:45information you put into AI,
6:45information you put into AI, generally the worse it gets.
6:48generally the worse it gets.
6:48generally the worse it gets. Obviously, there's a there's a sweet
6:50Obviously, there's a there's a sweet
6:50Obviously, there's a there's a sweet spot um around I think it's like 20% of
6:53spot um around I think it's like 20% of
6:53spot um around I think it's like 20% of the actual context window.
6:54the actual context window.
6:54the actual context window. Uh depends on every single model is
6:56Uh depends on every single model is
6:56Uh depends on every single model is different, but as a rough idea. So,
6:59different, but as a rough idea. So,
6:59different, but as a rough idea. So, instead of having all this crap where it
7:00instead of having all this crap where it
7:00instead of having all this crap where it just bloats that context window when
7:02just bloats that context window when
7:02just bloats that context window when things are crawling,
7:04things are crawling,
7:04things are crawling, you get this where it tells you exactly
7:06you get this where it tells you exactly
7:06you get this where it tells you exactly what you need to know. And more
7:08what you need to know. And more
7:08what you need to know. And more importantly with Vercel, they actually
7:10importantly with Vercel, they actually
7:10importantly with Vercel, they actually did a uh a little teardown of how this
7:12did a uh a little teardown of how this
7:12did a uh a little teardown of how this actually works.
7:14actually works.
7:14actually works. But the most interesting thing was that
7:16But the most interesting thing was that
7:16But the most interesting thing was that it was 3 KB,
7:18it was 3 KB,
7:18it was 3 KB, so a 99.37%
7:20so a 99.37%
7:20so a 99.37% reduction in payload size instead of 500
7:23reduction in payload size instead of 500
7:23reduction in payload size instead of 500 KB.
7:25KB.
7:25KB. This is super important. Um but what I
7:27This is super important. Um but what I
7:27This is super important. Um but what I will say, the pitfall that most people
7:29will say, the pitfall that most people
7:29will say, the pitfall that most people fall into if they are doing this for
7:30fall into if they are doing this for
7:30fall into if they are doing this for their own website is the fact that there
7:33their own website is the fact that there
7:33their own website is the fact that there isn't parity. So, if you get a 1 KB
7:36isn't parity. So, if you get a 1 KB
7:36isn't parity. So, if you get a 1 KB payload, but it actually only has like
7:38payload, but it actually only has like
7:38payload, but it actually only has like half of the information of the page,
7:40half of the information of the page,
7:40half of the information of the page, that's far worse than just using the
7:42that's far worse than just using the
7:42that's far worse than just using the original HTML. So, what you've got to
7:44original HTML. So, what you've got to
7:44original HTML. So, what you've got to make sure that you do is that the
7:46make sure that you do is that the
7:46make sure that you do is that the markdown version
7:48markdown version
7:48markdown version is in parity with the HTML version. And
7:51is in parity with the HTML version. And
7:51is in parity with the HTML version. And in some cases, you need to expand
7:53in some cases, you need to expand
7:53in some cases, you need to expand things. So, like for example, an FAQ
7:54things. So, like for example, an FAQ
7:55things. So, like for example, an FAQ section, if it's inside of a component,
7:57section, if it's inside of a component,
7:57section, if it's inside of a component, then you need to make sure that that FAQ
7:59then you need to make sure that that FAQ
7:59then you need to make sure that that FAQ actually gets brought into markdown in a
8:01actually gets brought into markdown in a
8:01actually gets brought into markdown in a way that's nice.
8:03way that's nice.
8:03way that's nice. There's loads of intricacies depending
8:04There's loads of intricacies depending
8:05There's loads of intricacies depending on what you're doing here, and I can
8:06on what you're doing here, and I can
8:06on what you're doing here, and I can talk your ear off from all the different
8:08talk your ear off from all the different
8:08talk your ear off from all the different ways that you can do this. But, the best
8:10ways that you can do this. But, the best
8:10ways that you can do this. But, the best way to do this is just giving us a
8:12way to do this is just giving us a
8:12way to do this is just giving us a shout, and we can talk you through it.
8:14shout, and we can talk you through it.
8:14shout, and we can talk you through it. Finally, let me show you a couple more
8:16Finally, let me show you a couple more
8:16Finally, let me show you a couple more things. Um so, this is Tetris. This is
8:19things. Um so, this is Tetris. This is
8:19things. Um so, this is Tetris. This is Wikipedia. I'm sure you've seen this
8:21Wikipedia. I'm sure you've seen this
8:21Wikipedia. I'm sure you've seen this before. However, something you probably
8:22before. However, something you probably
8:22before. However, something you probably never thought about is why did Wikipedia
8:25never thought about is why did Wikipedia
8:25never thought about is why did Wikipedia end up doing so well in the first place?
8:27end up doing so well in the first place?
8:28end up doing so well in the first place? And why does it appear so high on
8:29And why does it appear so high on
8:29And why does it appear so high on Google? And the reason being, one of the
8:31Google? And the reason being, one of the
8:31Google? And the reason being, one of the many reasons being here, is that if you
8:34many reasons being here, is that if you
8:34many reasons being here, is that if you look at this on the left-hand side, this
8:37look at this on the left-hand side, this
8:37look at this on the left-hand side, this is a table of contents. And if you look,
8:39is a table of contents. And if you look,
8:39is a table of contents. And if you look, we have an unordered list, and then we
8:42we have an unordered list, and then we
8:42we have an unordered list, and then we have the list items. And you can see
8:43have the list items. And you can see
8:43have the list items. And you can see table of content. So, list item, list
8:46table of content. So, list item, list
8:46table of content. So, list item, list item, list item. It's all semantically
8:49item, list item. It's all semantically
8:49item, list item. It's all semantically correct, which basically just means it's
8:50correct, which basically just means it's
8:50correct, which basically just means it's it's the way that HTML should be
8:52it's the way that HTML should be
8:52it's the way that HTML should be written. It's the way that
8:54written. It's the way that
8:54written. It's the way that computers can easily interpret what's
8:55computers can easily interpret what's
8:56computers can easily interpret what's actually going on on the screen.
8:57actually going on on the screen.
8:57actually going on on the screen. And there's not a tremendous amount of
8:59And there's not a tremendous amount of
8:59And there's not a tremendous amount of JavaScript being fired here. So,
9:01JavaScript being fired here. So,
9:01JavaScript being fired here. So, whenever we're building something like a
9:02whenever we're building something like a
9:02whenever we're building something like a blog, you're going to make sure that you
9:04blog, you're going to make sure that you
9:04blog, you're going to make sure that you have a table of contents that matches,
9:06have a table of contents that matches,
9:06have a table of contents that matches, kind of similar to Wikipedia, but you
9:08kind of similar to Wikipedia, but you
9:08kind of similar to Wikipedia, but you know, very very similarly, you don't
9:10know, very very similarly, you don't
9:10know, very very similarly, you don't necessarily want to have all the
9:11necessarily want to have all the
9:11necessarily want to have all the JavaScript, or it should still run and
9:13JavaScript, or it should still run and
9:13JavaScript, or it should still run and be semantically correct without
9:15be semantically correct without
9:15be semantically correct without JavaScript. So, all of these different
9:17JavaScript. So, all of these different
9:17JavaScript. So, all of these different little bits and pieces add up. Um last
9:20little bits and pieces add up. Um last
9:20little bits and pieces add up. Um last one I want to show you as well is if we
9:22one I want to show you as well is if we
9:22one I want to show you as well is if we go back into here. So, again, we're
9:23go back into here. So, again, we're
9:23go back into here. So, again, we're doing stuff that's not dev. Um we're
9:25doing stuff that's not dev. Um we're
9:25doing stuff that's not dev. Um we're going to right-click. We're going to hit
9:27going to right-click. We're going to hit
9:27going to right-click. We're going to hit inspect. Don't need a developer involved
9:29inspect. Don't need a developer involved
9:29inspect. Don't need a developer involved to do this. You just literally click
9:31to do this. You just literally click
9:31to do this. You just literally click into this thing on the left. Doesn't
9:32into this thing on the left. Doesn't
9:32into this thing on the left. Doesn't matter where you click. Try not to click
9:34matter where you click. Try not to click
9:34matter where you click. Try not to click a a link. Just try to click anywhere in
9:36a a link. Just try to click anywhere in
9:36a a link. Just try to click anywhere in here, and press command F or control F
9:38here, and press command F or control F
9:38here, and press command F or control F if you're on Windows and type JSON, j s
9:42if you're on Windows and type JSON, j s
9:42if you're on Windows and type JSON, j s o n.
9:43o n.
9:43o n. Now, what you're looking for
9:45Now, what you're looking for
9:45Now, what you're looking for is JSON-LD on the page. So, let's have a
9:48is JSON-LD on the page. So, let's have a
9:48is JSON-LD on the page. So, let's have a quick look. This isn't what I'm looking
9:49quick look. This isn't what I'm looking
9:49quick look. This isn't what I'm looking for. Let me have a look.
9:51for. Let me have a look.
9:51for. Let me have a look. This is what I'm looking for. You see
9:53This is what I'm looking for. You see
9:53This is what I'm looking for. You see breadcrumb JSON-LD. So, pop this open
9:56breadcrumb JSON-LD. So, pop this open
9:56breadcrumb JSON-LD. So, pop this open and you can see the information about
9:57and you can see the information about
9:57and you can see the information about the breadcrumb inside of here.
9:59the breadcrumb inside of here.
9:59the breadcrumb inside of here. Um let's have a look for another one.
10:01Um let's have a look for another one.
10:01Um let's have a look for another one. Okay.
10:03Okay.
10:03Okay. Collection page JSON-LD for blog. Okay,
10:06Collection page JSON-LD for blog. Okay,
10:06Collection page JSON-LD for blog. Okay, great. This is showing me what's
10:08great. This is showing me what's
10:08great. This is showing me what's actually on the page from a JSON-LD
10:09actually on the page from a JSON-LD
10:10actually on the page from a JSON-LD perspective. I understand what's going
10:11perspective. I understand what's going
10:11perspective. I understand what's going on here and by proxy, a computer
10:13on here and by proxy, a computer
10:13on here and by proxy, a computer understands it, too.
10:15understands it, too.
10:15understands it, too. So, this is really important uh to show
10:18So, this is really important uh to show
10:18So, this is really important uh to show essentially that you know what's going
10:20essentially that you know what's going
10:20essentially that you know what's going on on the page and you're presenting
10:21on on the page and you're presenting
10:21on on the page and you're presenting that to Google. You're presenting that
10:22that to Google. You're presenting that
10:22that to Google. You're presenting that to all these different AI crawlers. It's
10:24to all these different AI crawlers. It's
10:24to all these different AI crawlers. It's really helpful to give some rich
10:26really helpful to give some rich
10:26really helpful to give some rich structured data.
10:29structured data.
10:29structured data. Final thing that's worth doing here is
10:31Final thing that's worth doing here is
10:31Final thing that's worth doing here is there is a JSON validator from Google.
10:34there is a JSON validator from Google.
10:34there is a JSON validator from Google. Um I won't show it in this session in
10:36Um I won't show it in this session in
10:36Um I won't show it in this session in particular, but you should always just
10:37particular, but you should always just
10:37particular, but you should always just check at one level that the JSON is
10:41check at one level that the JSON is
10:41check at one level that the JSON is valid. Um I know there's a couple in
10:43valid. Um I know there's a couple in
10:43valid. Um I know there's a couple in here where we need to do a couple of
10:44here where we need to do a couple of
10:44here where we need to do a couple of bits of tweaking to make sure that the
10:45bits of tweaking to make sure that the
10:45bits of tweaking to make sure that the JSON is valid, but it's always important
10:48JSON is valid, but it's always important
10:48JSON is valid, but it's always important to do that. And we can talk a little bit
10:49to do that. And we can talk a little bit
10:50to do that. And we can talk a little bit more about that in the future if it's
10:51more about that in the future if it's
10:51more about that in the future if it's particularly interesting to you.
10:53particularly interesting to you.
10:53particularly interesting to you. But, I hope this gives you a rough idea
10:55But, I hope this gives you a rough idea
10:55But, I hope this gives you a rough idea of everything you can get started with.
10:56of everything you can get started with.
10:56of everything you can get started with. This is just kind of like the the
10:58This is just kind of like the the
10:58This is just kind of like the the scratching the surface of what AEO, SEO,
11:02scratching the surface of what AEO, SEO,
11:02scratching the surface of what AEO, SEO, uh you know, all of these different
11:03uh you know, all of these different
11:03uh you know, all of these different functionalities. Uh but, what I want to
11:05functionalities. Uh but, what I want to
11:05functionalities. Uh but, what I want to do is is have more deep dives about this
11:08do is is have more deep dives about this
11:08do is is have more deep dives about this and talk more about it in the future if
11:10and talk more about it in the future if
11:10and talk more about it in the future if you're particularly interested. Um
11:12you're particularly interested. Um
11:12you're particularly interested. Um appreciate you watching this the whole
11:13appreciate you watching this the whole
11:13appreciate you watching this the whole way through. Have a great day. Take
11:15way through. Have a great day. Take
11:15way through. Have a great day. Take care.
11:15care.
11:15care. >> [music]
11:16>> [music]
11:16>> [music] >> Bye-bye.
Questions answered
Editorial answers from the original Roboto page.
What's the difference between AEO, GEO, and SEO?
They're all basically the same thing despite the different names people use. What matters is that they work. These techniques help your website get discovered and ranked by search engines and AI crawlers, and you can apply them yourself or work with an agency if you want help building them out.
Read the original ↗How can I see what Google actually sees on my website?
Right-click on any page and hit inspect. Then press command, shift, and P on Mac or control, shift, and P on Windows, then type disable JavaScript. Refresh the page and you'll see what crawlers see without JavaScript running. This shows you roughly what Google indexes from your site.
Read the original ↗Why does it matter if my website works without JavaScript?
Most crawlers, especially AI crawlers, don't run JavaScript on websites. If your content and navigation only work with JavaScript enabled, crawlers miss important pages and information. The more functionality you can deliver without JavaScript, the better your website indexes overall. Static content also gets cached better, which helps performance and discoverability.
Read the original ↗What is content negotiation and why should I care?
Content negotiation lets AI crawlers see your website in a format they can actually read, like markdown instead of raw HTML. When you add accept text markdown to your content, you give AI a clean, stripped-down version that's around 99% smaller in file size. This prevents wasting the AI's context window with unnecessary code and helps it understand your content better.
Read the original ↗How do I make sure my markdown version is good enough?
You need parity between your HTML version and markdown version. If your markdown only has half the information, it's worse than using the original HTML. Make sure everything on the page, including components like FAQs, gets included in markdown so AI crawlers see the full picture.
Read the original ↗What's RunRepeat doing right that other sites miss?
RunRepeat gets cited a lot because everything on the page is unique, interesting, numerical information that AI can't find anywhere else. If you create original data and insights that AI can't pull from other sources, you'll rank and get cited much better than sites with generic, copycat content.
Read the original ↗Why should I add JSON-LD structured data to my pages?
JSON-LD tells Google and AI crawlers exactly what's on your page in a language they understand. Adding breadcrumbs, collection pages, and other structured data helps you show that you know what you're doing and makes it easier for machines to interpret your content correctly. You can validate your JSON using Google's validator to make sure it's correct.
Read the original ↗Scope and context
- Transcript speakers have not been individually verified.