Update version.txt

fix fused_qkv model accuracy issue (deepspeedai#5217)

Fused_qkv model can not correctly choose the fused_qkv type. Need to
update the module_name_matches.

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Add script to check for `--extra-index-url` (deepspeedai#5184)

Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>

Switch cpu-inference workflow from --extra-index-url to --index-url (d…

…eepspeedai#5182)

This switch should cause no impact to the workflow, but ensures that we
only download this package from the correct feed rather than
defaulting/falling back to the index-url
[default](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-i)
(PyPI) for if a package existed with a higher version there that would
be chosen instead.

Remove optimizer step on initialization (deepspeedai#5104)

All ZeRO 1/2/3 stages call the optimizer's `step()` on its
initialization. This increments a counter in the optimizer and produces
a different result in parameter update with the normal usage of PyTorch.
This PR eliminates `step()` in the initialization and lazily configures
some internal states (linking *hp_params*) after the first `step()`
call.

---------

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Refactor the Qwen positional emebdding config code (deepspeedai#4955)

follow PR deepspeedai#4920 on Qwen inference code

Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>

Update release.yml

Mixtral FastGen Support (deepspeedai#4828)

Adds support for Mixtral with FastGen. Key features implemented:

1. Top-2 MoE support
2. Better support for RoPE thetas
3. The mistral model implementation

---------

Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>

Fix 4649 (deepspeedai#4650)

Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>

Add safetensors support (deepspeedai#4659)

Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>