Deny Checkout when order limit is exceeded

[Guest]

I've written some custom code to modify the shopping_cart.php file to check if the customer has ordered within a certain limit. EG. The users orders 5 of a product and the limit is three the customer gets an alert to tell them to change their order. The problem I have is that if the customer doesn't change their order they can still checkout. I want to make so if the customer clicks on checkout when their order is too big they get sent back to the shopping cart. If any one has any ideas it would be greatly appreciated.