Skip to content

Commit

Permalink
Don't call ibv_qp_to_qp_ex when hardware does not support extended QP
Browse files Browse the repository at this point in the history
The sizeof 'struct ibv_qp' allocated by ibv_create_qp is 160.
If the memory holds the 'struct ibv_qp' was allocated at the
upper boundary of a memory page, cast it to 'struct verbs_qp',
whose size is 360, may across the memory page boundary. It will
trigger invalid memory access to the next memory page.

The issue can be reproduced with OPA and QIB HCA.

For example run over OPA:
 Server Node: $ ib_read_bw -F -N -n 1000 -u 20 -q 257 -s 4194304
 Client Node: $ ib_read_bw -F -N -n 1000 -u 20 -q 257 -s 4194304 <sever>

 Program received signal SIGSEGV, Segmentation fault.
 ibv_qp_to_qp_ex (qp=0x5555557a5f10) at libibverbs/verbs.c:624
 624             if (vqp->comp_mask & VERBS_QP_EX)
 (gdb) bt
 #0  ibv_qp_to_qp_ex (qp=0x5555557a5f10) at libibverbs/verbs.c:624
 #1  0x000055555556af4a in create_reg_qp_main (ctx=ctx@entry=0x7fffffffd500, user_param=user_param@entry=0x7fffffffd670, i=i@entry=21, num_of_qps=num_of_qps@entry=128)  at src/perftest_resources.c:1597
 #2  0x000055555556b6d7 in create_qp_main (num_of_qps=<optimized out>, i=21,  user_param=0x7fffffffd670, ctx=0x7fffffffd500) at src/perftest_resources.c:1613
 #3  ctx_init (ctx=0x7fffffffd500, user_param=0x7fffffffd670) at src/perftest_resources.c:1552
 #4  0x0000555555558e9c in main (argc=<optimized out>, argv=<optimized out>) at  src/read_bw.c:149

624             if (vqp->comp_mask & VERBS_QP_EX)
(gdb) p qp
$1 = (struct ibv_qp *) 0x5555557a5f10
(gdb) p vqp
$2 = (struct verbs_qp *) 0x5555557a5f10
(gdb) p *qp
$3 = {context = 0x55555578ad00, qp_context = 0x0, ....
(gdb) p *vqp
Cannot access memory at address 0x5555557a6000

Signed-off-by: Honggang Li <[email protected]>
  • Loading branch information
Honggang-LI committed Jul 24, 2020
1 parent 4de1710 commit d30e91e
Showing 1 changed file with 8 additions and 6 deletions.
14 changes: 8 additions & 6 deletions src/perftest_resources.c
Original file line number Diff line number Diff line change
Expand Up @@ -1649,14 +1649,16 @@ int create_reg_qp_main(struct pingpong_context *ctx,
return FAILURE;
}
#ifdef HAVE_IBV_WR_API
ctx->qpx[i] = ibv_qp_to_qp_ex(ctx->qp[i]);
#ifdef HAVE_MLX5DV
if (user_param->connection_type == DC)
{
ctx->dv_qp[i] = mlx5dv_qp_ex_from_ibv_qp_ex(ctx->qpx[i]);
if (!user_param->use_old_post_send) {
ctx->qpx[i] = ibv_qp_to_qp_ex(ctx->qp[i]);
#ifdef HAVE_MLX5DV
if (user_param->connection_type == DC)
{
ctx->dv_qp[i] = mlx5dv_qp_ex_from_ibv_qp_ex(ctx->qpx[i]);
}
#endif
}
#endif
#endif

return SUCCESS;
}
Expand Down

0 comments on commit d30e91e

Please sign in to comment.