We present quad-layered structural color filters producing transmissive red (R), green (G), and blue (B) colors with high brightness and high purity, where thicknesses of layers for the RGB colors are optimized by using a L-BFGS-B algorithm. To evaluate the performance of the proposed structural color filters, computer-based inverse designs based on meta-heuristic and reinforcement learning algorithms are employed, where the optical properties obtained from the inverse designs are comparable to those shown in our proposed design. A peak separation phenomenon in dual cavities is applied to make a spectral response rectangular, and also a resonance order is optimally tailored to maximize the transmittance at a resonant wavelength with the suppression of undesired higher-order resonances at the same time for achieving pure colors. Transmission efficiency over 75% and the full width at half-maximum (FWHM) less than 90 nm are achieved. Besides, selecting a cavity medium with a high refractive index allows the optical properties of the structural color filters to remain almost constant in wavelength over a broad range of incident angles up to 60°. Moreover, only a few deposition steps are necessary, thus leading to a much simple fabrication as compared to previous works that involve a series of complicated lithographic processes. The approach described in this study may provide new ways for achieving diverse applications, such as displays, imaging devices, decorations, and colored solar cells.