The discovery and optimization of materials still remains a significant challenge when dealing with a very large feature space, limited data, and, if the experiments and/or calculations are expensive to perform. This paper presents intelligent sampling methods designed to guide experiments or computations towards user-defined specific regions, termed "regions of interest," within vast and complex feature spaces. The focus of this work is to compare several adaptive sampling methodologies to identify 50 optimized Al/CuO thermite materials that meet user specifications, while minimizing the number of samples to reduce experimental costs. We considered Bayesian optimization and active learning techniques, both driven by specific learning schemes, to guide the sampling task. Particularly, we introduced two variations of the original ParEGO algorithm and evaluated their effectiveness in sampling optimized materials within the whole feature space against active learning methods. This work showed that, using a limited initial dataset of 100 points, the active learning approach is more effective to navigate in a vast design space as it leverages uncertainties and predictions from a surrogate model, combined with an acquisition function that prioritizes decision-making on unexplored data.