As has been noted previously, you need to let your lump burn till it's burning clean (pretty clear smoke) for new lump allow 20 - 30 minutes. I always give myself at least a half-hour between the time I light the fire and when I put the food on.

Chicken thighs are one of my favorite things to cook. That said, chicken in general is susceptible to picking up smokey flavor and it also tends to contribute to (bad)smoke through the drippings. My family doesn't like smoke flavor in most things.

The firestarters may cause your temp to rise hotter at first while they're burning, but then after they're out and it's only lump it usually drops back a little before it goes up again. And as Stike said, the fire starters smoke like hell if they go out before burning completely. You don't need to leave your dome open while your firestarters are burning. In fact if you close it, that will start the draft feeding the fire the way it would during the cook so you'll be starting the lump you'll be burning for the cook and getting the bad smoke out of there.

Keeping the daisy wheel on the chimney will retain the smoke in the cooking area. This is good if you are trying to "get smoky" but bad if you don't want smoke. For chicken (and all cooks over 300) I leave the daisy cap off all together. You can control the temp using the lower draft door only. For 350, the door opening could be anywhere from 1/4" to the thickness of a nickel, depending on burning characteristics of lump, new vs old or combo, or if you went past your target temp when getting the coals started.

Another thing to note is that for cooking chicken parts, I either go indirect with something to catch the drippings or go direct using a raised grid.