Can't start a service without service.py

[dmsfabiano]

I am quite new to bentoml so I may be missing something from the documentation that I have read. Is there no way to start a Service programmatically without a service.py?!

I have the following:

class BentoMLServer(Server):

    @classmethod
    def serve(
        cls,
        path_to_bento: Path,
        host: str,
        port: int,
        reload: bool,
        serving_level: ServingLevel,
        model_version: str,
        profiling: bool,
    ) -> None:
        mlmodel = BentoMLPackager.load(path_to_bento)
        app = cls.get_fastapi_app(
            model=mlmodel,
            serving_level=serving_level,
            model_version=model_version,
            profiling=profiling,
        )
        bento_model = BentoMLPackager.get()
        runner = bento_model.to_runner()

        svc = Service(f"Citiflow_{mlmodel.id}", runners=[runner], models=[bento_model])
        svc.mount_asgi_app(app)

        serve_http_development(
            path_to_bento,
            working_dir=os.path.expanduser(path_to_bento),
            port=port,
            host=host,
)

I have my custom BentoMLPackager which does the standard you would expect from any framework say framework.sklearn, and I have encapsulated my framework on this get_fastapi_app which returns an standard FastAPI app

I have followed the source code enough to find the svc.mount_asgi_app and serve_http_development methods. However, I seem to be kind of stuck, because I cannot find any way to actually save the service I have created,

serve_http_development fails with Failed to import module service: No module named service but I would prefer not to maintain a service.py as I am wrapping bento around my home-grown framework

[parano]

Hi @dmsfabiano, unfortunately BentoML relies on the import path of "bentoml.Service" instance, e.g. "service:svc", to be able to load the service definition across multiple API servers and Runner processes.

We've helped many users with similar situations where BentoML needs to integrate with an internal framework. Happy to hop on a quick call to learn more about your setup, understand what BentoMLPackager does, and discuss alternatives. Feel free to ping me in the community slack!