Code with Brylie - e47

gistfile1.txt

(...) Hello and welcome to an open source live code hangout.

(...)

We'll be working on the Western Friend website today.

(...)

I've got an issue here.

(...)

It's a small optimization I suppose.

(...)

Basically we have keywords that are shared across our content and they allow people to navigate our site based on common topics or these keywords or essentially topics to find related content essentially.

(...)

I implemented the keywords feature. I just created a link to the search page that would put that in the query string. So essentially every keyword tag on a page like article or media item will link to our search page.

(...)

Our search page is one of the slowest and least perhaps optimized or at least as slow

(...)

pages in the site. I believe it's due to the nature of needing to search across all the content types. It's probably optimized about as much as it can be. Also I didn't write the code. It comes from the Wagtail Core search. So I'm sure they put a lot of thought and design into that.

(...)

So since the search page is a little bit out of bounds from our project perspective I'm not really managing the indexes there or the queries.

(...)

I'd rather just reduce the reliance on that. I've done this in a previous pull request where we were redirecting 404s to the search page. The main reason is we're getting a lot of bot traffic and it's kind of costing us a bit of excessive fees. For example we're using an open source analytics provider to collect anonymous visitor analytics

(...)

and in one month we exceeded our yearly quota.(...) It's just bot traffic and I've put some stuff there to mitigate bot traffic but I guess these are legitimate bots. I think it's a crawler. It's like Microsoft and Google or just for some reason stuck in a crawling loop on our site. I don't know how to stop that. It's a bit frustrating but nonetheless at least we'll try to reduce the load by creating a more efficient display for the keywords. Maybe that'll also break the bots out of their crawling loop.

(...)

I've put CloudFlare and proxy in place, reverse proxy or whatever and traffic hasn't diminished but it has identified a lot of traffic and that's now reflecting in our analytics provider who sent us a billing email that we'd already exceeded our yearly quota in one month which is like 600,000 hits. That's quite a lot. We don't get that many visitors, human visitors impressions.

(...)

So let's get to it.

(...)

I believe what we can look at here is essentially the name of our search view.

(...)

We should only encourage users to search when they're wanting to search, not lean heavily on this unfortunately. I think it's a really good solution to when somebody clicks on a keyword just to search for all the content with that keyword in it. We're in there.

(...)

I think it's leading to these issues and somehow the bots are getting stuck and they're hammering our site and I don't think these are malicious bots. It's weird.

(...)

So what I can do is get this search view name and we'll look for,(...) you know because this is just using the internal Wagtail search with the OR operator which is also kind of heavy but that was necessary.

(...)

It used to be AND but it wasn't, I can't remember exactly what was happening because we would get multi key queries and multi word queries and we want to fuzzy search for any of those.

(...)

So let's just remind ourselves of that.

(...)

So with this search view,(...) I guess this is defined in our core URLs so that it appears at a top level not a name space or not a sub URL.

(...)

So we should have a search.

(...)

Yeah and I gave it the name search so that's going to be a bit difficult but let's look for pi.

(...)

Oh wait actually HTML.

(...)

Yeah so here's an example of the 404. This is where actually we used to redirect the 404 but now we give them a search field, search form.

(...)

But I think if I grab this URL search, we'll get a little closer to them after. Just to see where we're redirecting the URL search.(...) So action complete, you can now log in. Oh yeah search, I need that part of it.

(...)

Our nav bar has a search which is cool search and here's where it is. So the tags, query tag, what I'm going to need to do I think is just create a tag display.

(...)

So we have this tags model which is basically from Django tag it so we're not actually managing this as well.

(...)

But it turns out we'll need a view for that.

(...)

This is sort of a, well actually the Molly Wingate page might have given us a good approach here.

(...)

Page tags right. We have page tags and if I'm using tags consistently then we should be able to get pages with the tags and tag page tags something like that.

(...)

Let's give it a try.

(...)

So creating list view.

(...)

I don't always remember these imports.

(...)

So this is a little bit easier.

(...)

It has changed so the context is a bit out of date but at least we've got that now.

(...)

Page page. So these are the tags.

(...)

Pachinator.

(...)

Yes that's pretty good. So really helpful if this works.

(...)

And this tags page list is going to be very similar to our search page because it's essentially a search view which is why I use that. It has multiple content types. It has at least two content types.

(...)

So we need this.

(...)

Base.

(...)

And then what are we doing here? So.

(...)

Yeah that's the right thing. Let's see if it suggests anything else. But now it's alright.

(...)

So we will need to set up a URL structure for it. Yeah it can just be tags slash.

(...)

Tag actually.

(...)

I'll just forget what's the boilerplate for URLs right.

(...)

Alright. I don't know if this is. I forget why this is there but let's go ahead and do it.

(...)

URL patterns.

(...)

Looks good I think that should be.

(...)

I can't remember if these tags have spaces in them.

(...)

So they won't necessarily be slugified.

(...)

Well.

(...)

Let's just test it out one step at a time. So if we go to our core URLs again.

(...)

I should be able to just put in any arbitrary tag.

(...)

[sniff] [piano music]

(...)

We can clean that comment up, it's not what we're doing in this project, so... [piano music] Good to go there, so yeah if I run a server...

(...)

[typing] [cough] Excuse me.

(...)

[sniff] It's that time of year.

(...)

[piano music]

(...)

Okay, so... Postgrade's not running. [typing]

(...)

Uh, this is actually in DevPy.

(...)

[typing] Start the database.

(...)

And run this, I'll be right back. Just make myself less sneezy.

(...)

[piano music]

(...)

[piano music]

(...)

[piano music]

(...)

Alright, good to go.

(...)

Server's running.(...) So we should be able to go now to localhost and tags,(...) slash test.

(...)

[clears throat] [piano music] Alright, so first error...

(...)

keyword tags.

(...)

So that's coming from the URL.

(...)

And I think the main thing is that the URL...(...) this is how generators are, so just be careful. Tag...

(...)

is coming in from the URL.

(...)

Tag...

(...)

[piano music]

(...)

And then this is the problem. So first I just want to...

(...)

get the page to render, then we'll come back to here.

(...)

Sorry, okay.(...) Just need to get rid of that method for just a second.

(...)

And run that.

(...)

Missing a query, so, yep.

(...)

Okay, well let's just...(...) [piano music]

(...)

This is the right thing. Okay, so we got that. So the thing's wired up, that's some good news.

(...)

But here's the right thing, I'm going to have some troubles. Already having troubles.

(...)

Pages,(...) the top level Wagtail Pages don't have a tags property. I've got some page models that inherit from this,(...) where we've defined tags and...

(...)

when we get this error, we'll take a look at it again.(...) This is looking for properties on the top level Wagtail Page Model.

(...)

All of these, content type ID, well, deep archive index page depth, that's actually the event.

(...)

All of these are sort of... These are properties or reverse names.

(...)

Let me just see how the tags are implemented.

(...)

But they won't be inherited or linked to the page model. So where are we using tags? Well, Molly Wingate blog was one, so that was on WFPage's model.

(...)

Molly Wingate blog, index page... Molly Wingate blog.

(...)

WFPage tag.

(...)

Tag item base.

(...)

This is why I used the search, in fact.

(...)

But here we've got tags, cluster level tag manager through, with a through model. I wonder if that through is necessary.

(...)

Query set.

(...)

Well, basically what I can do as a minimum thing is, can you perform operations on a query set?

(...)

And then,(...) since I can't probably do one query,

(...)

so to take the union of query sets,

(...)

what would be ideal solution would be that I could just search for page objects based on this tag field.

(...)

If I could see,(...) even though this inherits from page,

(...)

the search isn't looking for transient, it's not transient, it's not looking for fields on child models, and I believe I've asked this in the Wagtail support before.

(...)

So this may be a Django thing. Let's check out Chadgbt.

(...)

Can't help but it's out here.

(...)

So I have several Django models with a common tags field. We'll call them models A and B.

(...)

I like to filter instances of those models based on the value of tags field and return the common query set, a unified to a view. And the idea is I don't want to make a bunch of queries here because I believe this is also going to be hit by the bots and I've been trying to learn how to reduce our database load because I think the database is one of the main remaining blocking points in the Django call chain. We're not yet switching this asynchronous code. I don't know if it'll be practical for this project.

(...)

Okay, so we'll need some advanced query, Django queries, iter tools, and chain.

(...)

So to filter instances across multiple Django models by a common field like tags and return a unified query set,(...) you can use Django's Q objects for advanced queries and iter tools chain for combining query sets. Okay, so it looks like we'll start multiple queries, we'll have multiple query sets, but at least we can combine them. I'll settle for that, that's cool. So we'll import Q and Django DB models from Django DB models and chain from iter tools. Okay, let's get our imports in there.

(...)

And iter tools is core, so it goes to the top. I'm trying to sort of keep our important order wagtails like on the level of Django imports.

(...)

And then we'll set up a Q object for your filter tags, I contains.

(...)

And essentially, oh yeah, so it's an incentive contains and this brings me back to one important point is are these tags slugs or will they be potentially multiple words? If they're multiple words, then that's going to mean all bets are off a bit.

(...)

And how would multiple word tags be handled? Hey,(...) let me look for an example of that.

(...)

So I'm just trying to think how to set this up. Basically,(...) so our server is running and we'll come back to this error. But if I go to the Wagtail admin, I don't know if I've got a Django,

(...)

a super user.

(...)

I do, I do, I do, I do, I do, I do. We have an article.

(...)

And this article should allow me to tag it with our common tags. I hope I did that right.(...) Tags, comma separated lists of tags, multi word tags with spaces will automatically be enclosed in double quotes. Okay, so there's a, there's a problem.

(...)

Might require a migration.

(...)

And I think I can, I can publish this real quick and actually take a look at our tags and see because we've done a big import multi word, vegetarian and so we're at the QBS FGC J. So we've got a few of these Baird Rust and all right. So unfortunately, it's not a slide. A slide. A slug.

(...)

The tag is on a slug.

(...)

I don't know if we would, we want them to be case sensitive because some of these are like proper nouns. Others are general concepts.

(...)

But anyway, I'm not enforcing, I'm not the capitalization police over here. I'm just, I'm just making this work. They'll use them how they want. They can have punctuation even.

(...)

So even if I wanted to, how would I make a view for this tag?

(...)

Well, it's going to be HTML escaped, right? That's the key realization. So it's not going to be a slug.

(...)

That means our view needs our URL and possibly view needs. Where the heck my URL and go, man.

(...)

All right.

(...)

So it's just this ring, which I think by default is string, but I'll just annotate that here.

(...)

So and now that I think of it, we're using Django tag. I was thinking tagify slug it. What's it called again? Let's check the documentation.

(...)

Okay.

(...)

Tags and forms, text and admin, tag API. I'm looking for URLs just to see how they annotate the URL.(...) So we do, we do allow them to be a case. They are case sensitive and and they are slugified. Oh, oh.

(...)

That's cool. That's good to know.

(...)

Can I have an example view?

(...)

Oh my goodness.(...) Just take me to the like tutorial. How do I use it?

(...)

I mean, this is sort of how I use how I add it to a model. I'll tag one minute. How do I get in a view? Come on. Come on. Ballcats.

(...)

The power of ballcat is with you. All right. We can use them in the admin.

(...)

And Django resty. I'm not using that. It's got a serial analyzer there.

(...)

The API. That's cool. So we can add them. We can remove them. We can set them. We can find similar.

(...)

Crazy.

(...)

Wow. I wonder how that works.

(...)

Slugs.

(...)

You green and juicy. Hey, that's it though. No need.

(...)

No need. So I can filter tags. Slug.

(...)

Or name in delicioso.

(...)

This is coming along. Now the thing is it'll be HTML escaped and then I need to unescape it or what's the inverse. Yeah.(...) Yeah. Yeah.

(...)

And on this note I need to take a small break. I'll be right back.

(...)

Sorry for the delay.

(...)

All right. So I think at this point we should turn to the terminal or the shell.

(...)

So we've already found our list of tags here and Central America is at the start.

(...)

The key is.

(...)

We know we can't filter directly on the page of it, right?

(...)

All right, this is what happens.

(...)

Central America.

(...)

We get an error.

(...)

Page in.

(...)

There we go.

(...)

Yes, we get those errors. It's not a field. You can't do that.

(...)

So we're needing to target the specific models and then we can filter those models. That's cool. We can do that. Problem is I don't know where this is used and which model it's used.(...) Probably the magazine article, but it could also be a media item.

(...)

So what we need to do is just bring in our other models.

(...)

Really?

(...)

Magazine article, I think.

(...)

You know, you don't know. Your own code is a thing. It happens.

(...)

Interesting.

(...)

Step away for one week. I can't remember anything.

(...)

There's just too much to remember. We know it's not an expectation.(...) Open editors don't care about the editors, man.

(...)

Do I have a typo there?

(...)

Yes, I have a typo there.

(...)

I just don't like typing the MAGA.

(...)

I think that's the problem.

(...)

Okay, so now I think I can take this same query just to test it out real quick.

(...)

We get nothing back, but the search works. So good. From media models.(...) Where is the library model? Yes, library.

(...)

Don't know our own project code.

(...)

Now we'll find library items filtered by the tag.

(...)

Latin America,(...) Central America. Well.

(...)

Library items.

(...)

Not defined.

(...)

Library item.

(...)

Yes, that's correct because models are singular. Hey,(...) now we've got an item. So this will work.(...) The thing is, it's all going to be HTML escaped if I just print it to the console.

(...)

Cool. That means...

(...)

Doctrine of discovery.

(...)

Sounds ominous.

(...)

I just need to take a step back and look at how it's coming into the view.

(...)

So if I look for this item,(...) welcome strangers. How about we do that? Let's do that.

(...)

I'll wire it to the template and then sort of work my way forward through there.(...) In other words, I'll put the keyword into the URL for the new tag page and see what the context looks like.

(...)

Probably don't have to do so much guesswork. Anyway, let's just experiment with that. Welcome strangers. Shoot. Welcome.

(...)

I forgot.

(...)

It's our page and new library items.

(...)

What the ding dong?

(...)

That's so weird.

(...)

My short-form memory has just failed on me.

(...)

Welcome mean. Ah, this is a problem.

(...)

I guess of non-fuzzy search. Welcoming strangers, right?

(...)

We're all strangers.

(...)

On planet Earth. I need to remember that nobody is native.

(...)

We're all native, in other words. In a way. Eh, I don't want to get any political stuff going on.

(...)

But we need to kind of stop the violence and stop discriminating against one another as my own, your home.

(...)

This is our collective home. Yeah, so we've got the item here. We've got some topics, but topics are different than tags. Here are the tags. Social justice, Central America, refugees. Yeah, this is the main thing where I might have come. You know,(...) getting at it. We had a lot of refugees coming in to Finland, for example, and discrimination against refugees. And other issues such as austerity coming up and making things more complicated. It's not the refugees fault. The austerity measures are being imposed on people.

(...)

Alright, so we're able to do this. Now, what I can take a look at is the library item template.

(...)

And what we do in that template is render the tags.

(...)

In the loop, we might not need to use this specific anymore.

(...)

Medium time period. I think we put the tag sort of at the bottom because they're the most generic. So here are tags and instead of going to the search, I want to go to.

(...)

Tags that tag with the tag as the argument.

(...)

I think that should be correct. And we don't need them to query set anymore.

(...)

So if I go and I view this, I think live, I don't view it live status. I think if I just click this button, it'll open a new tab.(...) Tags that tag is not found tags. I'm talking about a value function. Patterning through the namespace.

(...)

Tag page. Right, right.(...) I suppose so.

(...)

Makes sense. So namespace is correct, but the view was not.

(...)

Okay. All right.

(...)

Maybe not.

(...)

What are our global URLs?

(...)

Here's our tag URLs.(...) Model, magazine.

(...)

See if I don't get namespace should work.

(...)

The dot is colon. I think.(...) Are you all together?

(...)

So view, which is relating to the model and enclose the muscles. Right.(...) And what do you use to separate?

(...)

A space like that.

(...)

There we go.

(...)

Yes. Now I've got the thing viewing rendering and.

(...)

I'm going to use the slug. Okay, so this is automatically using this slugified one. So, oh, that's so good.

(...)

Yes, that means I can go back to my tag URLs and put this back on slug.

(...)

And then. Wow. This is, I'm so glad this just works. This is your details that I wouldn't want to work out. Wow.

(...)

Those are topics. Oh, shoot. Tags are here.

(...)

Okay. So now they're not.

(...)

But I think if I just do tag dot slug.

(...)

I hope that the tag it module. Slugifies them, right? Oh, it works. It works. Okay.

(...)

Yes. Good. And.

(...)

Now we're back to the original error because in the viewer still looking at.(...) Page.

(...)

That's cool. So we actually don't need this model, but we're going to take list to start with just the library items because we know that that works first and then we'll add other models with page.

(...)

Import.

(...)

A library item there.

(...)

Now we should get some results.

(...)

To refresh pages, tags, social justice.

(...)

All right. So we're off to the races. We have a crease and I just haven't done anything that templates are rent render that. But it looks like what we need to do. I guess these creates that are lazy. So.

(...)

There may be some.

(...)

Optimizations going on.

(...)

I think that's be the slug though. I just noticed that.

(...)

For now we're just going to retain his library items. I'm just preparing them for the next. Okay. So. We're going to do. We're going to do. We're going to do. We're going to do. We're just going to retain his library items. I'm just preparing. I'm just setting up so we can have multiple query sets and then use the.

(...)

Well, actually that's different.

(...)

This is cool. So the filter.

(...)

Condition I contain so.

(...)

Here we don't need to do case insensitive search. I think we can just look for the slugs.

(...)

We'll just use a common filter condition.

(...)

Yes, it's just the same.

(...)

So I was hoping it'd be consistent like that.

(...)

So library items and what we can do is sort of get the code from the search page. There's a template.

(...)

And it has these partials.

(...)

Okay. So let me just think better if we need all of this, but we have these specific search templates and perhaps we can reuse those, but at a minimum viable level. I don't quite need those. We're okay. We're going to link to their and in the view I've got these things and then they're going to be just query set. I guess what will the name of that be items.

(...)

Can I change the context of the. And we'll just call this.

(...)

Items.

(...)

All right. So let's see if we can just list them out.

(...)

They all should have a.

(...)

Title.

(...)

And perhaps a.

(...)

An intro or teaser, but this is where we kind of get into the territory of needing. Specific templates like to render these items. They don't all behave the same. So we'll just go ahead and do that. Specific templates like to render these items. They don't all behave the same. So let me come back to this. I'm displaying the content and I'm not sure if you are allowed to get page URL in here from the wag tail tags.(...) Yours for Django models and page URL. So we're using Wagtail. So I'm using Wagtail. All right. I can include. What is it?

(...)

Wagtail.

(...)

Tags.

(...)

I think that's what it is. And.

(...)

This is actually page URL.

(...)

All right. Let's see if we get some errors here. If I've got this loaded correctly and each item should have a title because they're wagged. So.

(...)

All right. Let's see if we get some errors here. If I've got this loaded correctly and each item should have a title because they're wagged pages.

(...)

Whether or not they should be age two and other questions about markup and how to display the sort of teaser. We'll leave aside for now. All right. Wagtail tags not registered Wagtail core tags. All right.

(...)

Yeah. You know we don't remember these things. There's a lot of these. Cool.(...) OK. Hey it worked.

(...)

Yeah.

(...)

Social justice. OK there's a lot of heavy heavy reading here. It's working so far that.

(...)

We'll improve the display but let's go ahead and commit. So we've got an end to end sort of initial or like I would say a minimum viable thing so I can click on any.(...) Tag and I come to the tags page and we get some results back.

(...)

These aren't mixed results so let's let's say for the minimum viable let's get the mixed results.

(...)

In other words mixed your items that are tagged. I think the quickest way to find the models with the tags field would be to search the Python.

(...)

Perhaps for tags space equals.

(...)

Or or well I don't know. I don't always.

(...)

Yeah that were clusterable tagable manager cluster tagable manager your magazine news WF pages in the library. So first we've done the library. Let's go to the magazine.

(...)

Article model.

(...)

And come back over here.

(...)

To view and import that.

(...)

And. Then it looks like we've got.

(...)

News.

(...)

And it's probably news article at sea news item. Let's see.

(...)

Yeah.(...) GPT and Co-Pilot aren't really able to guess that they're not scanning my whole project whereas the language server internally is.

(...)

And in WF pages.

(...)

Models import to be a page. Those are all good.

(...)

So we'll just kind of. Repeat this in the query set we got the filter condition which is common to all of them because they used a common field this is an important thing to note about being consistent.

(...)

That consistency can pay off in ways when you try to unify things or integrate things facing a lot of consistency issues and my day job because we've sort of re implemented very similar. Features in models and separate parts of our application and subtly different ways so the different names and behaviors and it's leading to a lot of bugs complexity confusion and we're trying to kind of consolidate and remember that hey we're in a monolithic application we have one core project. We don't need so much different behaviors even though people are interfacing with it through different API endpoints and even gRPC and things like that. So consistency is really important.

(...)

You don't kind of know it until you sort of have con I guess far enough down the line to see oh man. I've made three library items with magazine items. We'll get news items and we'll get pages this is really helpful see that now that I've got all this context in the imports and the initial filter condition to find chat GPT is able to really infer all this now we'll do the Django magic with the queue.

(...)

I say chat GPT I believe that go power is powered by that so we get all of our query sets and then we combine them. All right.

(...)

Now this is interesting.

(...)

How handle pagination here.

(...)

Let's see if our combined crazy works.

(...)

For getting the errors here.

(...)

No no errors OK no new content so let's see refugees I'm just going to see if there's now we've got a couple of things we had a library item and.

(...)

The library media.

(...)

Social justice are these all library.

(...)

We make it a little bit bigger I can see these euros scanning scanning scanning.

(...)

Well what's a tag that's common to.

(...)

Page and library item let's go ahead and tag a page from his is also sorted and paginated. So I need to figure out what the paginator.

(...)

In a list view what is the name of the paginator object.

(...)

Let's view template.

(...)

Stuff like that I couldn't really ask on Stack Overflow even though it's very Stack Overflow thing you know or reading the docs of Django would take me a while but it's right here and I just get the answer back and no criticism about how I asked it or anything like that. Or is it a duplicate I just sort of get an answer paginator number pages greater than one they will introduce our pagination. And I think I just have this reusable paginator HTML we've seen it used here and the search page so let's just hop back over there. I think I closed the search index.

(...)

Let's close this I'm thinking good with the models now.

(...)

And library item. We're done with that pretty much we can open it back up pretty much don't our URLs everything is wired up.(...) We're just now in our views and our template and we're going to put a little pagination magic in a template so. I can reference how this search one did it. We have a common paginator and I just do this and I need this L. If no items.

(...)

We'll copy that and read it more closely here.

(...)

And for else.

(...)

You know just in case there are no pages tagged with that I think people might somehow end up there putting random things.

(...)

Or search crawlers might try a bunch of things.

(...)

Breathing back to this thing how do I mitigate this issue with so many. Search it's. So many bots crawling our thing is paginated items equal.

(...)

And we're not using query string in this view this was a custom thing to preserve the query string across pages.

(...)

Let's see if this works if this paginated items expects a paginator I think so I think so I could just call it paginator.

(...)

Paginator equals paginator that would have been more consistent I can fix that here.

(...)

If that's what it expects otherwise it would be paginator dot something.

(...)

All right so.

(...)

This I don't think we need now we're good.

(...)

I didn't see any change paginator here just right.

(...)

Show up here.

(...)

Paginators so that's not happening.

(...)

First check is is paginator dot HTML rendering.

(...)

So like kind of print debugging here.

(...)

But I'm going to go to the pagination. template.(...) And just pop in.

(...)

To OK there we go it's not rendering.

(...)

And.

(...)

Why.

(...)

Let's try it without this.

(...)

If items. Render them like a loop over them that's happening.

(...)

And this is displaying but this is not include paginator HTML or paginator.

(...)

HTML is in the kind of global namespace templates anything directly in the templates directory will be paginator.(...) Otherwise I would put in a subdirectory pagination for example.

(...)

I've saved the file.

(...)

It's not rendering.

(...)

Oh sorry sorry.

(...)

It is rendering but this condition is not met. OK.

(...)

So that rendered.

(...)

But there's no other pages.

(...)

There's exactly 10 items.

(...)

That's a coincidence.

(...)

One.

(...)

So this is kind of telling me that.

(...)

Items should be passed in.

(...)

That makes sense and that's kind of why I called it here.(...) What I called it here I mean.

(...)

Why I called it what I called it.

(...)

Intuitively.

(...)

But this is in my mind a query set but I guess it's a paginated query set.

(...)

So print debugging approach is not helpful.

(...)

You know I don't want to do this so I just want to see that we can get some results.

(...)

So there is that.

(...)

So what's happening is I've got an internal method called get paginated items that I've designed and it's not aligned it's not consistent with the way the list.

(...)

View.

(...)

Provides pagination so I can't it's not reusable. So this is an example of consistency where I haven't I've got some bespoke code that's not consistent with the Django framework and I can't just use code.

(...)

Generated by the framework such as this paginated.

(...)

Query set.

(...)

Pagination I'll show you this real quick so I'll refresh my brain that way as well.

(...)

Your paginated items takes a query set.

(...)

And returns paginator.

(...)

And with an ellipsis to omit certain pages so that you have the full paginator context. You know there's twenty three pages however many and you can go to the first five and then he lies the values between it. This is standard Django stuff.

(...)

But.

(...)

My interface here is bespoke I made this data class.

(...)

And the interface might clearly doesn't. Align with.

(...)

The Django paginator so what does the Django paginator look like.

(...)

More succinct that's not really the word more brief more tickless space but creating a paginator and Django template with the buttons for each page and ellipsis for longer page ranges involves a bit of logic in your template so we get a list view in paginated by and I should align my pagination code with this.(...) If paginated number of pages.

(...)

So ensure this view is set up for pagination like so this is the important part paginated by.

(...)

Then our template code we can create a pagination control the goal is to show the buttons for a few pages around the current page and use ellipsis to indicate skipped pages this is really good Taylor fit answer.

(...)

Okay so then if there's some pages greater than one we'll need a paginator.

(...)

And I think in my paginator logic I don't pass in the paginator I only pass.

(...)

Perhaps I only pass the paginator and not the page object or no it looks like if I look at the HTML.

(...)

Let's give it the list view so I'm going to really give the trying to get a very bespoke answer so here's my Django list view.

(...)

[MUSIC]

(...)

See if you can connect the dots there that are sort of implicit. So to use your shared paginator.html in your tagged page list view template,

(...)

you need to ensure that the paginator and the page object context variables are correctly passed to the paginator.html. These variables are automatically created by Django's list view. This is what I was mentioning earlier. I think I'm only passing the paginator,(...) but I thought it has a member pages, and then there's page object on that.

(...)

[MUSIC]

(...)

These variables are automatically created by Django's list view. [MUSIC] When using pagination. But in your case,(...) I was just trying to think, I'm overriding these probably unnecessarily here. This context object name can just be page object. [MUSIC]

(...)

Consistency, right? Consistency.

(...)

[MUSIC] I might be facing issues during to how I'm constructing the context. Here's how you can modify your view to ensure the paginator works correctly. Ensure that get query set returns a query set.

(...)

[MUSIC] That is returning a...

(...)

Ah, it's casting to a list. That was good. Catch, I didn't think about that.

(...)

Of course. So this will just return a loop.

(...)

Whoops, wrong button.

(...)

There we go.

(...)

Nice. And then they get context data.

(...)

[MUSIC]

(...)

And here I can use my shared thing. Okay, even if I'm inventing my own convention.

(...)

[MUSIC] At least I'm consistent with my own conventions.

(...)

[MUSIC]

(...)

So it looks like these are a little bit redundant.

(...)

[MUSIC]

(...)

Now, if I wanted, I would refactor my...

(...)

This is a bit of a tricky thing. If I want to get aligned with the Django internal APIs and how their pagination system works,

(...)

this would be the proper way to do it.

(...)

And these are already available.(...) So that would mean I just need to change.

(...)

The other places where I'm using the paginator to also adhere to that interface. Paginator equals paginator, pay job goes pay job.

(...)

Very consistent, very familiar for other developers. That would increase the scope of this issue.

(...)

[MUSIC]

(...)

As a first step...

(...)

[MUSIC] Yeah, it has git page method, but okay.

(...)

[MUSIC] And here's where I'll need to preserve the context argument.

(...)

[MUSIC]

(...)

This is cool, you can update the context. I've always just been treating it as a dictionary and then adding keys, but okay.

(...)

Maybe this is the update will override the keys. But if these are already provided by the view, then I don't need to override them.

(...)

I don't need to use items, that's just my own little thing here.

(...)

Suppose.

(...)

Hmm.

(...)

So,(...) you know, how lazy or what would the word be?(...) Pragmatic, am I going to be here?

(...)

The right thing is to really make the code consistent. It means changing this internally how it works. As you can see, I'm already pretty fuzzy about how it should work.

(...)

[MUSIC]

(...)

Plus I need this elided page range, which is why I developed this bespoke.

(...)

Well, what I can do.

(...)

[MUSIC]

(...)

Let's ask some help from my super friend here.

(...)

[MUSIC]

(...)

I'll do this in parallel. I'll call this paginator2. [MUSIC] So that I won't break things. You know some things like that.

(...)

Just a little bit of output here. Ah.

(...)

It already has the ellipsis, so I think this is a good move.

(...)

[MUSIC] Somehow it's already got the ellipsis. Without me having to do anything fancy there. Interesting.

(...)

Let's try it out. Okay.

(...)

If it works here in the library,(...) I mean in the tag page.

(...)

I'll look at it using it more globally. I want to have this one pagination template.

(...)

[MUSIC]

(...)

Paginary refactor.

(...)

And I believe then all I need to do, close this one for a minute here. Paginary refactor.

(...)

I'll need to come back to the view code.(...) I think everything will be there. So I've already done this and I just need these arguments with that and that. So get all the way to the width.

(...)

[MUSIC] There we go.

(...)

[MUSIC]

(...)

So let's see here what happens if we get an error here.(...) [MUSIC] [MUSIC]

(...)

So I see why it cast into a list earlier. [MUSIC] Oh, yeah, this needs to be a page out of it.

(...)

[MUSIC]

(...)

But that's not where the error is originally, of course.

(...)

I'm trying not to use that helper.

(...)

[MUSIC]

(...)

So I'm trying where this is originating.

(...)

So, we've got chain. So it says object type iter tools chain, right? That's being returned from our query sets.

(...)

Has no index length.(...) So something ain't quite right. Let me just see if I've got that correct.

(...)

[MUSIC]

(...)

Combined queries, how's that used?

(...)

[MUSIC] So I still have to do a little bit of manual preparation here, and I believe that's why I created this helper, in fact.

(...)

This isn't inherently a Django convention. This is us just perhaps overriding that.

(...)

[MUSIC]

(...)

So it brings me back to the pragmatic thing. So essentially, if I just use our helper and move forward with this, it's an example where I'm not perhaps following conventions, but they're not super clear what they are. It's a bit opaque to me, and I don't know if this is a convention.

(...)

[MUSIC]

(...)

And since I'm using this combined query set approach, it's a bit fuzzy to me.

(...)

[MUSIC] But this should just be a query set.

(...)

[LAUGH] Let me just paste this in there.

(...)

[MUSIC]

(...)

Actually, this is probably more. [MUSIC] It has more context,(...) more meaningful.

(...)

[MUSIC] Now, I didn't follow its instructions. There you're encountering type error object, but iteratools chain has no length. The originator expects an object that supports length.

(...)

Typically, a query set or a list. However, iteratools chain does not support length as it is a generator. To resolve the issue, you need to convert the chain query sets into a list before passing them into the pattern. However, keep in mind that this can be memory intensive if the query sets are large.(...) [MUSIC] So back to the list chain, but they're not a different problem. [MUSIC]

(...)

And then they just move the paginator.(...) I don't know, listen and get context. I'm still not doing this step. I don't really acknowledge that. I know it has been returned. [MUSIC]

(...)

That won't work. You can't call functions in the template. All right.

(...)

That's one Django design decision. So it's kind of hallucinating this a bit. This won't work. This is decided that it can't pass arguments into functions, at least.

(...)

You can call them without arguments so that you keep logic out of the templates, which is one of the biggest gotchas where things can get really complicated, the more logic you put in your templates.(...) All right. So let's just roll back here and go with my pagination helper. I've got a pagination, your paginated items.

(...)

Hopefully, so this will work with the chain query set.

(...)

Will it? [MUSIC]

(...)

Let's just try it. [MUSIC] So it needs to be created. I'll take the list and move that again.(...) Oops.

(...)

In our context, we're going to import this.

(...)

Yep. So we'll have to preserve this git argument in a moment. Let's just see if I can get this to work.

(...)

[MUSIC]

(...)

I'm using in search and I need to fix in the rendering of the items.

(...)

And I need to pass this query string in. All right, let's just do that now.

(...)

No, it's a little different, isn't it?

(...)

This context variable passed that in. Okay, yeah.

(...)

And I just called it search query string, but why don't we just call it query string?

(...)

Well, it's called paginated items.

(...)

Consistency.

(...)

And we'll call this query string.

(...)

We will need to look at these again.

(...)

Closing this.

(...)

Let's take one more look at this.

(...)

It needs to resemble the paginated items.

(...)

For item,(...) paginated items.

(...)

Can I do object list?

(...)

Page.

(...)

All right, I'm coming along here.

(...)

I will have to cast it to a list. It said it was CPU intensive, but it's necessary here. This is what I was a bit worried about. Because we're combining query sets across multiple tables, essentially, models,(...) things wouldn't work. Hey, pagination is working now. One or 15. So there's like 15 items. If I change the paginated by to like, let's say three, not 30.

(...)

Reload.

(...)

Let's see what happened here.

(...)

Dude.

(...)

There's such thin bar here. I can't even get to that.

(...)

I cannot hover over this with my mouse.

(...)

And there's five pages now and the items are rendering and I go to here. Yep, fine. So consistency.(...) And I noticed when it was paginated by one, our ellipses are working, aren't they? Wait for it to reload.

(...)

Yeah, there they are.

(...)

Pachinator.

(...)

Worked. Worked. So now I'm going to refactor.

(...)

I'm not sure what the conventions are again. I defined a convention here in a utility that we're reusing. So at least there is a convention in place.

(...)

And I think basically we want 10 items per page. So we should at least have two pages.

(...)

10 items is reasonable.

(...)

Now, whether this is performing, we'll have to keep track of page contains and the results because it's preserving the query string. So let's just take the query string off. Oh, but it's appending it. That was a, that's a bug.

(...)

All right. So watch this. Click here. It'll take me to page one and page two and now I have page one and page one and no problem.

(...)

But then, yeah, so you get a bug. It's like, what page am I on? It is a problem.

(...)

We can fix this. I'll change this. I don't need that anymore. We can fix this.

(...)

I probably have fixed it already on the search view.

(...)

Query string. I just reconstructed there. Makes sense. Query string equals page yada yada.

(...)

So it's a manual thing for the query string.

(...)

I did not.

(...)

Oh, I just write.

(...)

Okay, I see page number.

(...)

Let me thank you for a second.

(...)

No, I think that's correct.

(...)

[ Pause ]

(...)

Yeah, the solution is different because I need to pass in the page number. [ Pause ]

(...)

Let's not break that.

(...)

[ Pause ] A little cleaner that way. All right. So in the template, I think this is what I do.(...) In the paginator.(...) So we don't need to pass in the query string. It's just, that's actually the bug, I think.

(...)

Because it's already handling the pages correctly.

(...)

And, yeah, that works. So go to page two and page one. And there we go. And the end is just there. Okay. [ Pause ]

(...)

So, hey, there we are. We have a functional paginator using shared code.

(...)

There's a bit of a flash of white. These enhancements I'd like to do.

(...)

I can get rid of that flash. Perhaps with HTMLX. Or there's some modern browser APIs that can, for example, wait. So maybe the browsers are already trying to do this by default. Waiting until the background page loads. And then sending the content. But I do see a little flash of white. I'm not going to be really overly concerned about it to start a whole SPA movement by any means. I want to just find a little simple and local solution.(...) Keep sanity.

(...)

Very good. So now our audience and things and facets are working normally. We're using our search page less.

(...)

And essentially, this is one big old commit. Okay.

(...)

Initial.

(...)

[ Pause ]

(...)

Tags paginated.

(...)

Looks good. Reduced search page usage, but the goal is actually to create this paginated tags page.

(...)

We can find out other ways that the search page is being used. But I think this was a major one because from a crawler perspective, you know, I'm clicking on a page. I've got a list of, I've got an article. I'm crawling all the articles. And I'm taking to this, you know, each of these tags. I'm calling these tags.

(...)

And I'm going to put quotes around that. I just realized tags. Pages tagged extinction because this is not used in the sentence. Semantically, it's used by reference. We're referencing this way. So subtle difference. That means I'm referring to the word extinction, but I'm not using it grammatically or semantically.

(...)

Not sure if I'm really clear on that, but just grammar rule.

(...)

I don't know if I'm really describing this correctly, but the grammar you're referring to is known by use-mention distinction.

(...)

The rule is applied when a word or phrase is mentioned, but not used as a conventional semantic rule. So I guess I am kind of, I don't know, maybe it's hallucinating.

(...)

Essentially, when you're talking about a word itself as an object rather than using it for its meaning in a sentence, you put it in quotes.

(...)

So like here the tag extinction is an object. I'm not using the word extinction to apply any meaning other than the fact that we're using that word as a tag.

(...)

It's a really subtle thing.

(...)

This clarifies that you're referring to the word as a word or phrase as a phrase, not for the meaning and context.

(...)

So I'm referring to this whole phrase as just a phrase, and I'm not really meaning,(...) the exact meaning of these words. That's why it's so confusing.

(...)

In your example, page is tag extinction. The word extinction is mentioned, but not used in its usual semantic sense. It is not talking about actual extinction, but rather about the word extinction as a tag. Therefore, it is in closing quotes. The distinction is crucial in linguistics, philosophy, and sometimes in everyday communication to avoid ambiguity and make clear when a term is being discussed rather than used normally.

(...)

Really cool. Grammar can be interesting, but one of the challenges I'm having learning the language here in Finland, I need to learn Suwami, is that we're teaching people with grammar as a tool, as a pedagogical tool. In other words, grammar up front,(...) practice in the back.

(...)

I believe that practice comes first and grammar emerges out of practice, and people aren't ready to learn grammar until you're already familiar with the language, until you're already familiar using the language in daily life.

(...)

These grammar rules are tacit,(...) and making them explicit only becomes relevant in a much more advanced state. I know grammar is often used to help us categorize types of word usage and why there are differences in particular usage contexts, and to answer questions about why is it this way and why not that way. You need to refer to grammar rules, but I think it's misguided to align your whole curriculum or textbook around grammar concepts,(...) like partitative cases.

(...)

I find it really painful and a painful way of learning. It's not how humans naturally learn, so divorced from our natural means of language acquisition, which is just conversational using language in context and picking up the rules statistically, just through having a lot of exposure.

(...)

I don't know that I have a solution, how I would design a language course. I am teaching English here in Finland, and we don't really talk about grammar, we just talk about understanding. Do you understand? Can you understand? Usually, people are really flexible when I try to speak Swami, and our brains can parse irregular grammar and all sorts of things.

(...)

Brains are very flexible, and we can read into what people are trying to say. We can read between the lines, read through the mistakes. In any case, grammar is interesting, but it shouldn't be a pedagogical tool.(...) Pedagogical. Anyway,(...) at early stages, maybe B level or C level language learners should start learning

(...)

some more in-depth grammar.(...) We're going to find a template real quick.

(...)

Let's look at the top pages tag and put a couple of quotes around there.

(...)

Refresh.

(...)

Yeah, quote the tag.

(...)

And that way, especially with multi-word tags, the other thing I need to do is probably get the tag name, which might already be doing it here implicitly, but let me go to a multi-word tag, like climate change.

(...)

Page is tag climate name change. Okay, here's one.

(...)

Distinction. Another bug, so to speak.

(...)

When I click on this tag from a user perspective, I see the phrase climate change.

(...)

Here is climate. It's the slug, right?

(...)

So in my code, I need to distinguish in the view code between the quarg is already coming in there.

(...)

The slug is coming in there.(...) So it would be nice if I could get the tag name here from this tag slug. Let's do that.

(...)

I believe I just need to get the Django tag it tag model,(...) which I don't know where it is, but from tag models import tag, there we are.

(...)

Pop both of those in there.

(...)

Distinguish between them in the template.

(...)

I don't think I actually used the tag slug in the template. Let's just see.(...) There we are. Okay, so I can move one line of code here.

(...)

Yeah, that's just, you know, like nice touch.

(...)

And we can see the capitalization here and you know, it's not grammatical. It's just referencing that this is an object. This is a tag object.

(...)

And semantically in the context of the sentence, this is an object just arbitrary string, we could say.

(...)

It's not actually meaning climate change. The semantics of those two words are removed when I put in quotes.

(...)

Very cool. Let's just pop that in there.

(...)

Nothing else jumps off the page at me. Like improvement what? Oh, yeah, yeah, of course. Push this up here, then open a pull request.(...) The blobbiness and big header level of stuff. I don't like the red color, but that was an editorial decision. So I won't complain anymore about that.(...) It just looks bad. It looks like I visited all those pages. I haven't visited any of those.

(...)

Closes 10, 14.

(...)

Created draft here.(...) And what would be nicer way? Well, the problem I've been deferring just to get the page working is that these are perhaps all have different structure.

(...)

And on our search page,(...) we have different templates depending on the content type. Now we can kind of reuse those here.

(...)

But if I look at the specific search template with entity.

(...)

I look at my view.

(...)

Search template.

(...)

I think this is defined at the entity level.

(...)

Let's try this.

(...)

So while I'm iterating over them, I'll add an if statement here. And of course, we're not naming results, so I'll have to change the context name.

(...)

For item in there, do that.

(...)

And see if this just works.

(...)

It'd be pretty cool.(...) It did.(...) And then, yeah, so it's still blobby redness.

(...)

Almost looks worse, but in cards.

(...)

But at least they're in cards and the H2 is a bit heavy. At least it's consistent. So I'm basically back to the search page aesthetic.

(...)

And I think I'm going to go over this with Mary and see how she wants to change this and improve the search page.

(...)

So I'll leave it alone. I'll put the change that I currently did.

(...)

I'll commit that.

(...)

You know, because that's essentially how we're doing it. And if we do incorporate other content types, like article actually,(...) now that I look at it, it should be a little bit different.

(...)

So I wonder, okay, so this is climate change. If I can tag an article with climate change, what happens there? Do we get it in our search?

(...)

We're issued and,(...) oh, add article. This is bad.(...) This is only a conversation, but they're deprecating this model admin view instead of just fixing it.

(...)

Oh, actually it's just tagging these existing.

(...)

It has to be, it's case sensitive. So that's the one thing.

(...)

The live climate change.

(...)

A lot on climate change.

(...)

These are all library items.

(...)

So what I'm noticing is it didn't come up at all.

(...)

Topic library item.

(...)

Oh, because I went to search. Yeah.

(...)

Right.

(...)

And the test article is using, okay, the search page. We need to come back. Yeah. Okay. We're not done yet.

(...)

There it is. But what I'm thinking is happening here is the results are stratified. So first of all, the library items appear sorted,(...) not in any particular way.

(...)

Then all the articles appear, they pretty much appear.

(...)

It's okay. So at least the template thing is working.

(...)

They pretty much appear.

(...)

Library items, magazine articles, news items, pages. Stratified.

(...)

So what is our experience that we're after here? Which is why one query would be really better.

(...)

Yeah, it does exhaust. So it's going to iterate over each of those. That's what chain works. Makes sense.

(...)

I guess order order by

(...)

Let's see if it picks up on this. My cursor key lambda instance title lower.

(...)

Test article.

(...)

Let's see if it breaks.

(...)

So it is running.

(...)

Now at least it appears QRST. Yes, W.

(...)

All right, so a bit of work there, but not too bad.

(...)

A lot of this code is self-explanatory. These comments were mainly to prompt, copilot.

(...)

I will add an overall doc stream.

(...)

This is kind of a signal what I'm intending to do in this query set.

(...)

Anyway, whatever.

(...)

So these are actually separate.

(...)

Use the item search template available. What is this? Out of curiosity, what does this suggest? The magic wand. And the magic wand says,(...) not bad.

(...)

Some lint.

(...)

All right, so unit tests. I have to take a break,(...) but I believe I should at least put a unit test on the view,(...) get context, query set,(...) or yeah, get query set and get context. I think we just want to be notified if I make a change somehow that breaks something here.

(...)

That's what unit tests are good for. So I'll leave this pull request in a draft state.

(...)

We'll come back and work on in the evening.

(...)

So yeah, this has been another open source live code hangout.(...) I've been working on Western Friend website pull request number 1015.

(...)

You can check that out on GitHub.com, slash Western Friend, slash Western Friend.org.(...) I've got some CI issues.

(...)

I'll need to attend to as well. Deep source. Okay.

(...)

I'm not super enthusiastic about keeping deep source around. I might actually just disable it.

(...)

In any case, all right, thanks for stopping by. I hope you're doing well and have a great day.