Since the initial disclosure of Spectre and Meltdown, speculative execution attacks strategies have become one of the principal focus areas of security research. To defend against universal read gadgets in Spectre-PHT, there are proposals of using hardware extensions to enforce memory safety during speculative execution, thus helping protect victims from universal read gadgets. However, memory safety techniques cannot stop all Spectre-PHT variants, since not all variants rely on memory safety violations. Understanding gaps between memory safety and speculation safety are crucial for enhancing memory safety techniques to defend against all Spectre-PHT variants with low execution overhead. Therefore, we conduct a study on understanding residual Spectre-PHT vulnerabilities after applying memory safety techniques. We found that most residual vulnerabilities are mostly caused by type confusion and temporal memory safety violation.