the 2005 e46 individuals came out with an 18" wheel set. im sure that the 320D individuals in this year also had the 18's.
i think that you are more than fine with 18's. the torque on the diesel will more than make up for the weight and size of the wheels. just make sure that you go to that website, or check the rolling circumference on the wheels and that theyre within that few % allowance or else your wheel speed sensors might pick up incorrectly.
That pic that philip posted does show something else, you might want to think about lowering the car as well a bit, just so the wheels fit the arches a bit better.
And with the E90 wheels there is a slight difference with the offset compared to E46 wheels - tends to come out a bit more from the wheel arches than with the E46 wheels. So lowered suspension would help, especially with the 18" wheels I would think...
Interestingly we had my Wife's E46 (facelift) next to the gold E46 today and it seems like my Wife's car sits lower somehow. Doubt that it has a lowered suspension - maybe the shocks are a tad soft? I know that the gold car had new shocks fitted a while back...