A bug when running the demo inference on GPU
#5
by
HuggingLianWang
- opened
As shown in the figure, when asking the model the question "strawberry中有几个r?", it starts generating nonsense responses after outputting the "-s" characters.
As shown in the figure, when asking the model the question "strawberry中有几个r?", it starts generating nonsense responses after outputting the "-s" characters.
This is not a bug but an accuracy issue. The results in our README show the same output, which may be due to the overflow workaround. On CPU, this issue does not occur