Is there a bug?

[sunnyxiaohu]

Thanks for your wonderful works.
Is there a bug in:
https://github.com/ModelTC/QLLM/blob/main/models/int_llama_layer.py#L291

[Yiman-GO]

According to line40, 'attn_output = self.pv_matmul(attn_weights, value_states)' equals to 'attn_output = torch.matmul(attn_weights, value_states)'.