Boneless, skinless chicken breasts that are so common in grocery stores have very little fat, so they dry out unless cooked quickly.
I have more success with large split chicken breasts including the bone and skin. The larger pieces of meat are much more forgiving.
Rather than using olive oil on your chicken breasts, consider using Alabama white sauce, a mayonnaise based BBQ sauce. It can be used as a marinade, a basting sauce during the cook to retain moisture, a glaze at the end of the cook, and as dipping sauce at the table. It keeps the chicken moist and tender and provides a delightful flavor.
Lillie's makes an Ivory BBQ sauce that is one version of an Alabama white sauce, but I prefer the sauce I made from scratch using recipes online. When you make your own, you can adjust ingredients to your own preferences.