I believe this will not be possible without writing a custom shader. With a custom shader you would be able to do some custom logic, such as using the alpha value of Vertex colors as a “mask” and having something like this:
vertexColor.r = (1 - vertexColor.a ) * vertexColor.r + (vertexColor.a * diffuseColor.r)
vertexColor.g = (1 - vertexColor.a ) * vertexColor.g + (vertexColor.a * diffuseColor.g)
vertexColor.b = (1 - vertexColor.a ) * vertexColor.b + (vertexColor.a * diffuseColor.b)
vertexColor.a = 1;
This way vertex that have alpha = 1 will be rendered using diffuse color and others will be rendered using vertex color.