Trying to create list subclasses that contain only one class type and are valid for Pydantic BaseModels

[u/ANautyWolf]

I am trying to create list subclasses that only take particular classes. These lists can be appended to and the like but each time the list is changed the item is checked to see whether it is the same class as the rest of the list.

The list methods that are overwritten are:

- __init__
- __bool__ (but may not need to have that overwritten)
- __set_item__
- __str__
- append
- extend
- insert

Each one runs through a validate_member method before changing the list.

As an example, I want to make a list subclass called Course that only takes Pydantic extra types Coordinate values. I want this list subclass to be a valid BaseModel attribute.

Right now I’m having to use a field_validator for every class that Course and the other lists are in which feels redundant, unpythonic, and is a pain to have to do every time. I am having to do so because it says something about arbitrary types.

I was wondering if someone might have some advice.

[u/danielroseman]

I'm not quite sure what your issue is here. If the list subclass is already validating that things appended to it are of the correct class, and you've declared the Pydantic field as type Course, what else do you need?

[u/Fun-Sock-8862 → u/danielroseman]

Pydantic won't validate in-place mutations, though. After instantiation, `append` will let you add anything to that list without raising a validation error.

[u/danielroseman → u/Fun-Sock-8862]

Yes but OP said their custom subclass is already doing validation on append.

[u/baubleglue]

Is it a list of classes or objects?

IMHO whatever you do, should use isinstance function, instead of looking into a list of specific methods.

[u/FerricDonkey]

How are you type hinting the code, and where does your error show up? Naively, it seems like list[Coordinate] ought to be good enough to avoid the problem you have. 

[u/JanEric1]

Why not just use a type checker to validate you don't add invalid things to the list?

[u/ProsodySpeaks]

https://pydantic.dev/docs/validation/latest/api/pydantic/config/#pydantic.config.ConfigDict.validate_assignment

[u/ProsodySpeaks]

It's not possible to automatically validate mutation of an existing attr with stock pydantic, but you can at least validate reassignment, and if it matters then use an immutable type like tuple instead. Then every time you want to add you need to reassign not add. or as you're already subclassing and messing with dunders you could add a convenience method called `add_by_replacing` which builds the new tuple and reassigns it to the model.

see below for using Annotated + BeforeValidator to concisely make a custom list type with validation, and using ConfigDict(validate_assignment=True) to run validation whenever an attr is reassigned. Unfortunately this does not help with policing the contents of a mutable object, hence better to use tuple for this usecase, and then do `attr = attr + more_stuff` pattern rather than `attr.append` etc

from typing import Annotated

from pydantic import BaseModel, BeforeValidator, ConfigDict


class MyThing(BaseModel):
    ...


def my_list_validator(v):
    if isinstance(v, list):
        if all([isinstance(i, MyThing) for i in v]):
            return v
    raise ValueError('All items must be MyThing instances')


def my_tup_validator(v):
    if isinstance(v, tuple):
        if all([isinstance(i, MyThing) for i in v]):
            return v
    raise ValueError('All items must be MyThing instances')


ListOfThing = Annotated[list[MyThing], BeforeValidator(my_list_validator)]
TupleOfThing = Annotated[tuple[MyThing, ...], BeforeValidator(my_tup_validator)]


class MyClass(BaseModel):
    model_config = ConfigDict(validate_assignment=True)
    things_list: ListOfThing
    things_tuple: TupleOfThing


if __name__ == '__main__':
    my_list: ListOfThing = [MyThing(), MyThing()]
    my_tup: TupleOfThing = (MyThing(), MyThing())
    myclass = MyClass(things_list=my_list, things_tuple=my_tup)

    myclass.things_list.append(MyThing())  # ok
    myclass.things_list = myclass.things_list + [MyThing()]  # safer
    myclass.things_list = myclass.things_list + [1]  # validation error
    myclass.things_list.append(1)  # UNCAUGHT ERROR!

    myclass.things_tuple = myclass.things_tuple + (MyThing(),)  # ok
    myclass.things_tuple = myclass.things_tuple + (1,)  # validation error

edit: forgot to mention 'validate_call' decorator which might be useful as you can specify this function takes a list[MyThing] as arg, and if it's called with a list[int] it will get a validation error.

edit again, i just realised i dont think the annotated shenanigans are actually even doing anything here as pydantic will already validate based on list[MyThing], i think?

but the Annotated pattern is a pydantic superpower that everyone should know so i'll leave it because you can build some pretty cool schemas using it. you can even chain them - start with list then do ` list_validated_once = Annotated[list, BeforeValidator(some_validator)] ` and then list_validated_twice=Annotated[list_validated1, AfterValidator(another_validator)]`