How Gatsby and Next.js really work (and what it means for speed and SEO)

Written by Sadek Drobi in Engineering on November 29,2018

This is a transcript of an (extremely technical but nonetheless interesting) conversation about Gatsby, Next.js, speed, SEO, caching, and other things between Sadek (the CEO of Prismic) and Gabe (a core engineer – at Prismic, too). Watch the video, or scan through the highlights below, or read the entire thing – or do none of these things. Do as you please.

Below:

  • With React, the problem for SEO is that Google doesn't always see the actual content
  • Another problem is how fast React loads websites
  • Server-side rendering with Gatsby and Next.js
  • What exactly happens when you type a URL into the browser
  • What happens when you click a link on a loaded web page
  • Gatsby works "the other way around": there are no routes
  • What are the advantages of using Next.js over Gatsby

Sadek: So, React is becoming a popular framework, and some people use it or want to use it for websites. A part of using it in website means that you need to care about things like SEO and how fast the website is because speed is very important for websites, and that's why we thought about exploring more of the subject and doing this video about it. So, yeah, what's your take on that?

With React, the problem for SEO is that Google doesn't always see the actual content

Gabe: So yeah, for sure. React is a great framework, but it is just a JavaScript framework, and the environment that kind of surrounds React is up to you. But the way a React app loads, there's several problems with it in just using it by itself, say on a static site. The first problem is SEO. So say you have the React app, and it's just normal. It fetches data from a CMS, for example, and you load it into a website.

Well, if you do use a tool called Fetch as Google, you'll actually see what Google sees, and what Google sees there is a blank page, and it sees a blank page because it's actually loading content from a source.

Sadek: So basically there's this tool from Google, so you can see the websites as they would look like if Google is looking at them for indexing them.

Gabe: Exactly.

Sadek: The fact that React is a JavaScript framework, it means that it will have its time for rendering, and Google will only see maybe the loading or the first phase before actually the website displaying or rendering, and that the problem for SEO is that Google doesn't see the actual content.

Gabe: Exactly, so it doesn't know how to index.

Another problem is how fast React loads websites

Gabe: Normally, you have a normal React app. Every time you load the page, you have to fetch from a CMS or wherever your data source is – perhaps all the way across the world.

Sadek: And actually, if you want to fetch different pieces of content, like say four. That means four requests, and that takes even more time, right?

Gabe: Exactly. For a request from the United States and back or wherever you are.

Sadek: And since it's on the browser, it means it's like ...

It depends where you are to the server that is serving that content. So if you are in Asia, that will be very long, or if the servers are in the States, and people are in Europe, that's also will take a lot of time, and that's not very good for websites.

Gabe: Exactly. So what if you could fetch your content from a web server? What if the web server did the fetching rather than your browser? Or for SEO purposes, what if the server did the rendering? So all Google saw was a fully fleshed HTML page.

Sadek: So that's what something like Node.js will do. But then you will not be ... So, Node.js, for instance, a server-side technology, not client side as React. But server-side technology would rather do all the fetching and then will give you the HTML right away. But people ... Then you can't use React in that case, which is like a ... You know, so you lose all the advantages of using React. But there are some frameworks or solutions out there to yet allow you to use React, but they kind of avoid the problems of SEO.

Server-side rendering with Gatsby and Next.js

Gabe: Right, so you have something called server-side rendering, which is something you could perform on your own. But two solutions that have been implemented right now are Gatsby.js and Next.js.

Sadek: We are here to talk about actually the differences between these two, right?

Gabe: Exactly.

Sadek: So, Gatsby and Next.js, and they are quite different, right? In their approach to the problem.

Gabe: Yeah.

You can think of Gatsby.js as a static site generator

Gabe: Actually, what it does is, pretend you're hosting your website in local development. When you type in "Gatsby build," Gatsby will look at all the pages of your website and say, "Okay, that's loaded, that's loaded, that's loaded. Let's just serve these as the actual end points of the website." We did all the loading beforehand. We're done here.

Sadek: Right.

Gabe: And everything can actually be served on GitHub pages. It can be started on a static host.

Sadek:

So basically, Gatsby will try to build everything upfront, like it will go through all the sources, all the content files, and it will generate pages for them. That's what it will do.

Gabe: That's right.

Sadek: So actually, it generates a static website, and then you host it on anywhere you want as static files.

Gabe: Exactly.

Sadek: Okay.

Gabe: Now with Gatsby, every time you want to change something. Say you're doing documentation with Gatsby. If you change one word in the documentation, you have to type "Gatsby build" again, and Gatsby has to look at all of your pages. Say if you have 100 documentation pages, that's not so bad.

But now say if you have 5,000 pages, every time you are to change a word on your website, your Gatsby build command might take another 20 minutes.

Sadek: I don't know if it's 20 minutes. But it will take some time, right?

Gabe: It depends linearly on the websites.

Sadek: It depends, it depends, it depends, and there are ways to optimize that, but we're not going through that in this video. But that's basically the mechanics of how Gatsby works, right?

Gabe: Yeah.

Next.js: automatic server-side rendering

Sadek: And then let's take the same overview for Next.js. How does it work?

Gabe: Next.js is kind of a different system. You can think of its core as automatic server-side rendering. Normally, if you want to implement server-side rendering, it's kind of a pain, and Next.js kind of has this framework for you to do it, and it's very neat and easy to set up.

Sadek:

So basically, it means Next.js will take your React app, and it would render it on the service instead of rendering it on the browser?

Gabe: Exactly, and you don't even have to worry about making server code. That's the beauty of it.

Sadek: So It ties it up, or you kind of almost code as it's a normal React app. There are some conventions to take care of there. But then it does all the work for you.

Gabe: Right. It even pre-fetches data that you fetch normally using Fetch. So it's not just any normal server-side renderer. It will fetch your CMS content or whatever you're getting.

Sadek: Right, so that's how ... Okay. So to that, then it means that you need to host it, and it's in a server. You need to have a server, Node.js server, right? For Next.js.

Gabe: You do, but you don't need to worry about it. You just need to host it on Node.js.

Sadek: Yeah, but you don't need to code the server-side app actually.

Gabe: Right.

What exactly happens when you type a URL into the browser

In Gatsby

Sadek: I want to explore two different cases, so that we have the exact differences between Next.js and Gatsby. Imagine I am ... So I'm just typing into my browser, a URL of the page, okay? I typed the first time. I never loaded that website before, and I'm typing that into the browser. In Gatsby, how does it work? It gives me what? What does it return?

Gabe: In Gatsby, you'll get an HTML file--

Sadek: Pre-rendered.

Gabe: Pre-rendered with the app, and you'll get a JavaScript file, which is the React app because React still functions on the website.

Sadek: Okay, and that's a file that exists somewhere.

So nothing happens there. Just it's serving a static file that's generated.

In Next.js

Sadek: And Next.js, if I do the same thing, I type for the first time a URL of a page. What happens actually?

Gabe:

In Next.js, when you request something, it goes to Node.js. Node.js pre-renders all the content on the server, and it serves it right down to you as if your browser had rendered itself.

Sadek: So basically what happens there in both cases, you will get the HTML ready, right? You don't run it on the client's side, on the browser, in both cases.

Gabe: Right.

Sadek: But the mechanics differ behind the scenes in the case of Gatsby and Next.js.

What happens when you click a link on a loaded web page

Sadek: Now, I want to explore another case. That's a second case. Imagine I'm already on that page that I loaded, and I click on a link. How does it happen in Gatsby? Do you have an idea of how that works in Gatsby?

In Gatsby

Gabe: In Gatsby, it will ... Since your page is pre-rendered ... Well, it's a React app.

Sadek: So it will function as a React app, so it will actually go and run it on the browser side?

Gabe: Exactly.

In Next.js

Sadek: And in Next.js?

Gabe: In Next.js, it does the same thing. It functions as a React app.

Sadek: So both happen. It starts in React App, so that's why it's for Google who will be always requesting-

Gabe: Initial fetch. You don't have to worry about sequent fetches with Google.

Sadek: Okay, that's why it works. On the other hand, if it's a user, then it will act as a React app, actually?

Gabe: Exactly.

Sadek:

There's something interesting though in Gatsby is that they build a source file of all the JSON that you need for your page so that you can do only one fetch to get everything, right?

Gabe: Yes.

Sadek: And that's the cool thing about Gatsby in this case.

Gabe: It's very cool. This generally can be a very massive JSON file, but what Gatsby has done is they've kind of forced GraphQL in order to only get the data that you need. So, any API that you try and fetch data from, you have to convert it into GraphQL before you can use it in Gatsby, and that's to kind of save space in the JSON file because it can be very big.

Gatsby works "the other way around": there are no routes

To sum up: Gatsby gets all the content and then generates pages for that. So it functions differently from for, let's say, PHP or Node.js or all these kind of technologies.

Sadek: Basically, you don't have routes. You'd rather say for each page what are you going to generate. Next.js functions as a React app, so with routes and all of that. It's just there is a convention to follow. You need to be careful about respecting that convention so that you can get things pre-rendered.

What are the advantages of using, for instance, Next.js over Gatsby, in that case?

Gabe: Right. So, Next.js is kind of overridable, you could say. You can override anything on the Node.js side of things. So you have access there. You have access to use GraphQL or not, if you want. It's very flexible. It doesn't force anything upon compile time, whereas Gatsby, whatever you're going to do, it needs to be convertible to a static site in GraphQL.

Sadek: I saw one thing that was so nice about Next is that if you end up wanting to refresh one page, you can even do it on the client's side. You don't even have to pre-render it or anything like that, right?

Gabe: Right.

Sadek: So, if you want to add more dynamism into your page, it's very easy. Even if it's about the content of the page, you still can update that very easy. Okay, well that's cool. That gives us an idea about that. And then hosting, too. Oh yeah, we didn't talk about another solution. Maybe we'll have to talk about it next time, but:

There are solutions like Netlify that would take any React app, and they will kind of cache the result. They pre-render it.

Gabe: Right. That's a solution there too. But caching is your own choice as well. In Next.js, you can implement your own cache.

————

Thoughts? Questions? Comments? Did we get something wrong? Tell us on Twitter.

Sadek Drobi

Sadek is the CEO of prismic.io and co-created the Play Framework. He also helped LinkedIn and Twitter to scale their architectures. He's an avid badminton player and Juventus fan.