FedML

What's Changed

New Features

• [CoreEngine/MLOps] Supported LLM record logging.
• [Serving] Made the inference backend for deepspeed work.
• [CoreEngine/DevOps] Made the public cloud server scheduled into specific nodes.
• [DevOps] Added the fedml light docker image and related documents.
• [DevOps] Built and pushed light docker images and related pipelines.
• [CoreEngine] Added timestamp when reporting system metrics.
• [DevOps] Made the serving k8s cluster work with the latest images and updated related chart files.
• [CoreEngine] Added the skip_log_model_net option for llm training.
• [CoreEngine/CrossSilo] Supported customized hierarchical cross-silo.
• [Serving] Created the default model config and readme file if the user did not provide any model config and readme options when creating a model card.
• [Serving] Allow users to customize their token for end point and inference.

Bug Fixes

• [CoreEngine] Made compatibility when opening subprocess on windows.
• [CoreEngine] Fixed the issue that MPI Mode does not have client rank -1.
• [CoreEngine] Set the python interpreter based on the current running python version.
• [CoreEngine] Fixed the issue that failed to verify the pip ssl certificate when checking OTA versions.
• [CrossDevice] Fixed issues where the test metrics are reported twice to MLOps and loss metrics are clipped to integers on the Beehive platform.
• [App] Fixed issues when installing flamby on the heart-disease app.
• [CoreEngine] Added handler when utf-8 cannot decode the output and error string.
• [App] Fixed scripts and requirements on the FedNLP app.
• [CoreEngine] Fixed issues whereFileExistsError triggered for all os.makedirs.
• [Serving] Changed the model url to open.fedml.ai.
• [Serving] Fixed the issue for OnnxExporterError and added Onnx as default dependent library when installing fedml.
• [Serving] Fixed the issue where the local package name is different from MLOps UI.

Enhancements

• [Serving] Establish container based on user's config and improve code readability.

What's Changed

New Features in 0.8.4

At FedML, our mission is to remove the friction and pain points of converting your ML & AI models from R&D into production-scale-distributed and federated training & serving via our no-code MLOps platform. FedML is happy to announce our update 0.8.4. This release is filled with new capabilities, bug fixes, and enhancements. A key announcement is the launch of FedLLM for simplifying & reducing the costs associated with training & serving large language models. You can read more about it on our blog post.

New Features

• [CoreEngine/MLOps] Launched FedLLM (Federated Large Language Model) for training and serving GitHub Blog

• [CoreEngine] Deployed Helm Charts to our repository for packaging and ease of deploying on Kubernetes https://github.com/FedML-AI/FedML/blob/master/installation/install_on_k8s/fedml-edge-client-server/fedml-server-deployment-latest.tgz https://github.com/FedML-AI/FedML/blob/master/installation/install_on_k8s/fedml-edge-client-server/fedml-client-deployment-latest.tgz

• [Documents] Refactored the devops and installation structures (devops for internal pipelines, installation for external users). https://github.com/FedML-AI/FedML/tree/master/installation

• [DevOps] Deployed a new fedml fedml-light docker image and related documents. DockerHub GitHub doc

• [DevOps] Built the light docker image to deploy to the k8s cluster, refined k8s related installation sections in the document. https://hub.docker.com/r/fedml/fedml-edge-client-server-light/tags

• [CoreEngine] Added support for multiple simultaneous training jobs when using our open source MLOPs commands.

• [CoreEngine] Improved training health monitoring and properly report failed status.

• [CoreEngine] Added APIs for enabling, disabling and querying client agent status. The APIs are as follows.

curl -XPOST http://localhost:40800/fedml/api/v2/disableAgent -d’{}' curl -XPOST http://localhost:40800/fedml/api/v2/enableAgent -d’{}' curl -XPOST http://localhost:40800/fedml/api/v2/queryAgentStatus -d’{}'

Bug Fixes

• [CoreEngine] Create distinct device ids when running multiple Docker containers to simulate multiple clients or silos on one machine. Now using the product id plus a random id as the device id

• [CoreEngine] Fixed a device assignment issue in get_torch_device in the distributed training mode.

• [Serving] Fixed the exceptions that occurred when recovering at startup after upgrading.

• [CoreEngine] Fixed the device id issue when running in the docker on MacOS.

• [App] Fixed the issue in the app fedprox + sage graph regression and graph clf.

• [App] Fixed an issue with the heart disease app failing when running in MLOps.

• [App] Fixed an issue with the heart disease app’s performance curve

• [App/Android] Enhanced Android starting/stopping mechanism and fixed the following issues:

Fixed status displays after stopping the run. When stopping a Run during a round that has not finished, the MNN process will remain in IDLE state (it was previously going OFFLINE). When stopping after a round is done, the training will now stop Python server TAG in the logs is not correct. Now you can easily find the server mentioned in logs

What's Changed

New Features

• [CoreEngine/MLOps] Introducing the FedML OTA (Over-the-Air) upgrade mechanism for the training platform and serving platform.
• [Documents] Added guidance for the OTA mechanism in the user guide document.

Bug Fixes

• [Serving] Fixed an issue where exceptions occurred when activating the model inference.
• [CoreEngine] Fixed an issue where aggregator exceptions occurred when running MPI scripts.
• [Documents] Fixed broken links in the user guide document.
• [CoreEngine] Checked if the current job is empty in the get_current_job_status api.
• [CoreEngine] Fixed a high CPU usage issue when the reload option was enabled in the client API.

Enhancements

• [Serving] Improved data syncing between Redis server and Sqlite database.
• [Serving] Implemented the use of triple elements (end point name/model name/model version) to identify each inference API request.
• [DevOps] Updated Jenkinsfile to automate the building and deployment of the model serving Docker to the K8s cluster.
• [Serving] Implemented the model monitor stop functionality when deactivating and deleting the model deployment.
• [Serving] Checked the status of the end point when recovering on startup.
• [CoreEngine] Refactored the OTA upgrade process for improved robustness.
• [CoreEngine] Attach logs to the new Run ID when initiating a new run or deploying a model.
• [CoreEngine] Refined upgrade status messages for enhanced clarity.

What's Changed

New Features

• [CoreEngine/MLOps] Refactor the entire serving platform to make it run more smoothly on the Kubernetes cluster.

Bug Fixes

• [Training] The training status is still running after training.
• [Training] Fixed the issue that the parrot platform can not collect and analyze metrics, events and logs.
• [CoreEngine] Make the device unique in the docker container.

Enhancements

• [CoreEngine/MLOps] Print log does not show on the MLOps distributed logging platform.
• [CoreEngine/MLOps] Use the bootstrap script to upgrade the version of FedML when we don't need to publish the pip package.