Skip to content

Conversation

@keeper-jie
Copy link

fix do not generate rotary_pos_emb and cu_seqlens npy file error

bug discription:

When I try to follow the readme file generate  rotary_pos_emb and cu_seqlens npy file, there is no file generate. So I fix it and it work as expected.

before

Traceback (most recent call last):
  File "/dataset/rknn-llm/examples/Qwen2-VL_Demo/export/export_vision.py", line 89, in <module>
    torch.onnx.export(model, pixel_values, args.savepath, opset_version=18)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/onnx/utils.py", line 516, in export
    _export(
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/onnx/utils.py", line 1612, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/onnx/utils.py", line 1134, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/onnx/utils.py", line 1010, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/onnx/utils.py", line 914, in _trace_and_get_graph_from_model
    trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/jit/_trace.py", line 1310, in _get_trace_graph
    outs = ONNXTracedModule(
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/jit/_trace.py", line 138, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/jit/_trace.py", line 129, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1522, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/dataset/rknn-llm/examples/Qwen2-VL_Demo/export/export_vision.py", line 52, in export_onnx
    feature = model.visual(flatten_patches)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1522, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/dataset/rknn-llm/examples/Qwen2-VL_Demo/export/export_vision.py", line 68, in tmp
    rotary_pos_emb = torch.from_numpy(np.load("./rotary_pos_emb.npy")).to(dtype=hidden_states.dtype, device=hidden_states.device)
  File "/root/miniconda3/envs/llm_310/lib/python3.10/site-packages/numpy/lib/npyio.py", line 427, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: './rotary_pos_emb.npy'

after

==========================================================
Generating the rotary_pos_emb and cu_seqlens done.
grid_thw tensor([[ 1, 28, 28]])
The attention layers in this model are transitioning from computing the RoPE embeddings internally through `rotary_pos_emb` (2D tensor of RoPE theta values), to using externally computed `position_embeddings` (Tuple of tensors, containing cos and sin). In v4.54 `rotary_pos_emb` will be removed and `position_embeddings` will be mandatory.

fix do not generate rotary_pos_emb and cu_seqlens npy file error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant