When I first got to Employment University, my department asked me to take on the seminar series. At the time it was a bit hodgepoged and disjunct so I think they expected me to invite a couple of people here and there and call it a day. I had, however, been in charge of a seminar series as a grad student, so the task wasn't particularly daunting and I quickly realized I could use it to my advantage.
I sent out a request within the department for suggested speakers, and as per expectation I only got a few. That gave me freedom to pretty much ask anyone I wanted to see give a talk. I made a list of all the heavy hitters in my field within my geographic "sphere of invitation" and started working through it. I knew I was going to do the seminar series for at least two years, so I was able to spread these talks out so it wasn't blatantly obvious what I was doing.
In the process of hosting some big name folks to the department I have had the opportunity to not only increase interest in my filed within my department, but also get on the radar of some key people from other institutions. This is paying dividends both at conferences when I get the opportunity to catch up with these people and meet friends of theirs, but also because people tend to return the favor and invite you for a seminar at their institution.
More recognition + more invited talks + more interesting (for me) talks in my department = win. It can be a pain in the ass sometimes, but coordinating the seminar series can have huge up side if you use it to your advantage.