Great Side Quests. Even with all this, it's easy to look at something like the Alchemy recipe book and say to yourself "I know I don't want to waste this much time." So Dragon Quest IX uses side quests effectively to both encourage you to grind as well as to teach you how to grind effectively.
While only a handful of gamers may start playing with the motivation that they want to uncover every single secret and achieve 100 percent of everything possibly, a much higher ratio of gamers will at least to do the side quests in front of their nose -- especially the side quests that unlock new classes like Rangers and Paladins instead of the Warriors and Priests you start with. It was a side quest that encouraged me to start hunting Metal Slimes, but it was the amount of experience I got from killing them that made me keep going back to the dungeon they dwell within.
Many of those quests have you grind for the same items used in Alchemy. Maybe you need five Grubby Bandages to do a quest, but if you can figure out how to get those, then the six you need to make some cloth for Alchemy doesn't seem like such a big deal.
Likewise, there are several quests that require high skill or class levels. Earlier quests in those chains offered some fantastic items like Rogue's Robes for Thieves, so it stands to reason that you'd want to do the later quests -- which means grinding to level 40 in every single different class, and while you're at it, you may as well get a character up to 100 in every single different kind of weapon.
Buying Items Is Less Efficient. As Dragon Quest IX starts, it's fairly easy to buy all of the items and equipment that you need with the money you get from normal exploration and progression. It doesn't take long before the balance of gold earned and cost-of-items starts to seriously favor tilt towards cost in all the wrong ways. You either have to make hard choices about which equipment you want to buy when you get into town, or you have to grind for gold.
Since many of the items you'd be buying can be built with Alchemy, why not grind for that instead? You'll also pick up the gold and the experience you want -- and hey, it just so happens that the balance between gold earned during this grind and cost of the Alchemical ingredients you have to buy is much better-balanced for the player. In other words, you'd probably be grinding anyway, unless you have something better to do. But in Dragon Quest IX, you probably don't have something better to do.
Not-So-Great Main Quest. In many JRPGs, the storyline is the main attraction -- or at least, the game's designers want you to think that it is, thanks to constant attempts at manipulation via unskippable cutscenes and conversations. In Dragon Quest IX, the storyline is so far down the priority list that you can happily play for 40 or 50 hours in the middle of the game without even trying to progress. You're free to run around trying to gain levels and Alchemize. And it always seems possible.
Everything Seems to be Within Reach. Very few of internal goals that you need to grind towards seem unreachable. You can look at your Alchemy book and think "Yeah, I just need to steal six of these Ice Crystals, pick up some Fresh Water by the waterfall, and buy some Garish Garb, and I can make three new items!"
Since what you need is well-documented within the game and you can seek out the places where the items (or experience) are most common, you can sit down and play for a few hours and make signficant, tangible progress towards those goals. Then you progress the plot, find a new town with new recipes and a new dungeon with new enemies who drop new items to complete those recipes with, all the while going up in level, and it doesn't feel like such a grind at all. It feels like it's the right way to play the game.
This last point is one of the most important, as later in the game, the system starts to feel ungainly. For example, you'll find a collection of recipes which require Ethereal Stones. Ethereal Stones need Enchanted Stones, which in turn demand Mystifying Mixtures. Those require three Cowpats each. I figured out that I needed 19 Mystifying Mixtures, which meant 57 piles of bovine feces! Upon calculating this, I noticed my motivation for 100% completion fading quickly. Still, by then I was hooked. I had the spreadsheet. I came back.
Game designers have been more intently focused on appealing to completists in recent years, thanks primarily to the rise of Achievements. However, simply adding extended goals to a game isn't necessarily enough to motivate the players. Dragon Quest IX demonstrates that the utilization of multiple different mechanics in a coherent system can create a more motivating game. None of those mechanics are new. Many of them aren't even new to the Dragon Quest series. It's their measured application and interactions with one another that make these mechanics so successful.