-
Notifications
You must be signed in to change notification settings - Fork 88
Open
Description
The p7_hmm_SetConsensus can be used to set the consensus for a P7_HMM object, either by reading from the HMM matrix directly, or from a sequence given as argument.
That function executes the following loop:
for (k = 1; k <= hmm->M; k++)
{
x = (sq ? sq->dsq[k] : esl_vec_FArgMax(hmm->mat[k], hmm->abc->K));
hmm->consensus[k] = ((hmm->mat[k][x] >= mthresh) ? toupper(hmm->abc->sym[x]) : tolower(hmm->abc->sym[x]));
}In there, x is a symbol from the HMM alphabet, and is used to index hmm->mat[k], meaning it is expected to be in the 0..K range. This is the case if x is obtained from the esl_vec_FArgMax call; however, if x is read directly from the sequence, it may be in the 0..Kp range due to degenerate symbols and/or special letters. In that case, the indexing of hmm->max[k] can result in out-of-bound access (and valgrind reports so). This may cause inconsistent lower/upper-case consensus to be created.
Metadata
Metadata
Assignees
Labels
No labels