diff --git a/.gitignore b/.gitignore index f62343d..3590b74 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ *.exe +!/pkg/scoop/shim.exe /spoon *.pprof diff --git a/README.md b/README.md index 0740167..294454d 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,9 @@ relying on the existing community work in form of buckets. * More thorough `scoop search` * Better performance (Varies from command to command) +* Behaviour changes + * `spoon install app@version` will now use an old manifest and hold the app, instead + of generating a manifest (destined to be buggy) * Additional features * Tab completion for commands, flags and packages * Common command aliases @@ -21,8 +24,13 @@ below. ## Breaking Changes +* No automatic `spoon update` calls during `install`, `download`, ... * The `--global` flag hasn't beren implemented anywhere and I am not planning to do so as of now. If there's demand in the future, I will consider. +* Only `kiennq/shim.exe` is supported for shimming + > The older shim formats were included in scoop for backwards compatibility + > reasons. The solution is probably to simply reinstall all currently + > installed packages via `scoop export` and `scoop import`. ## Manual Installation @@ -41,6 +49,16 @@ below. Note that self-updating is *NOT YET* possible. To update, please use `scoop update spoon` for now. +## Runtime dependencies + +While spoon is written in Golang, it has runtime dependencies needed for +installing. Rewriting those would provide little to no value and cost a lot of +value. + +* [shim.exe](https://github.com/kiennq/scoop-better-shimexe) - Included in + Binary - MIT/Unlicense +* ... TODO + ## CLI Progress Progress overview for scoop command implementations. This does NOT include spoon @@ -60,24 +78,24 @@ There are basically three levels of implementations (and the states inbetween): | ---------- | ------------------- | ------------------------------------------------------------------------ | | help | Native | | | search | Native | * Performance improvements
* JSON output
* Search configuration | -| install | Wrapper | | -| uninstall | Wrapper | * Terminate running processes | -| update | Partially Native | * Now invokes `status` after updating buckets | -| bucket | Partially Native | * `bucket rm` now supports multiple buckets to delete at once | +| download | Native | * Support for multiple apps to download at once | | cat | Native | * Alias `manifest`
* Allow getting specific manifest versions | | status | Native | * `--local` has been deleted (It's always local now)
* Shows outdated / installed things scoop didn't (due to bugs) | -| info | Wrapper | | | depends | Native (WIP) | * Adds `--reverse/-r` flag
* Prints an ASCII tree by default | +| update | Partially Native | * Now invokes `status` after updating buckets | +| bucket | Partially Native | * `bucket rm` now supports multiple buckets to delete at once | +| install | Native (WIP) | * Installing a specific version doesn't generate manifests anymore, but uses an old existing manifest and sets the installed app to `held`. | +| uninstall | Native (WIP) | * Terminate running processes | +| info | Wrapper | | +| shim | Planned Next | | +| unhold | Planned Next | | +| hold | Planned Next | | | list | | | -| hold | | | -| unhold | | | | reset | | | | cleanup | | | | create | | | -| shim | | | | which | | | | config | | | -| download | | | | cache | | | | prefix | | | | home | | | @@ -87,14 +105,3 @@ There are basically three levels of implementations (and the states inbetween): | virustotal | | | | alias | | | -## Search - -The search here does nothing fancy, it simply does an offline search of -buckets, just like what scoop does, but faster. Online search is not supported -as I deem it unnecessary. If you want to search the latest, simply run -`scoop update; spoon search `. - -The search command allows plain output and JSON output. This allows use with -tools such as `jq` or direct use in powershell via Powershells builtin -`ConvertFrom-Json`. - diff --git a/cmd/spoon/cat.go b/cmd/spoon/cat.go index 2f9f3db..b30604f 100644 --- a/cmd/spoon/cat.go +++ b/cmd/spoon/cat.go @@ -28,13 +28,13 @@ func catCmd() *cobra.Command { return fmt.Errorf("error getting default scoop: %w", err) } - app, err := defaultScoop.GetAvailableApp(args[0]) + app, err := defaultScoop.FindAvailableApp(args[0]) if err != nil { return fmt.Errorf("error finding app: %w", err) } if app == nil { - installedApp, err := defaultScoop.GetInstalledApp(args[0]) + installedApp, err := defaultScoop.FindInstalledApp(args[0]) if err != nil { return fmt.Errorf("error finding app: %w", err) } @@ -44,12 +44,18 @@ func catCmd() *cobra.Command { app = installedApp.App } - var reader io.ReadCloser + var reader io.Reader _, _, version := scoop.ParseAppIdentifier(args[0]) if version != "" { reader, err = app.ManifestForVersion(version) } else { - reader, err = os.Open(app.ManifestPath()) + fileReader, tempErr := os.Open(app.ManifestPath()) + if fileReader != nil { + defer fileReader.Close() + reader = fileReader + } else { + err = tempErr + } } if err != nil { diff --git a/cmd/spoon/complete.go b/cmd/spoon/complete.go index e9a4e4b..f4385b8 100644 --- a/cmd/spoon/complete.go +++ b/cmd/spoon/complete.go @@ -53,7 +53,7 @@ func autocompleteInstalled( return nil, cobra.ShellCompDirectiveNoFileComp } - apps, err := defaultScoop.GetInstalledApps() + apps, err := defaultScoop.InstalledApps() if err != nil { return nil, cobra.ShellCompDirectiveNoFileComp } diff --git a/cmd/spoon/depends.go b/cmd/spoon/depends.go index b463ff0..8bf4179 100644 --- a/cmd/spoon/depends.go +++ b/cmd/spoon/depends.go @@ -27,7 +27,7 @@ func dependsCmd() *cobra.Command { if err != nil { return fmt.Errorf("error getting default scoop: %w", err) } - app, err := defaultScoop.GetAvailableApp(args[0]) + app, err := defaultScoop.FindAvailableApp(args[0]) if err != nil { return fmt.Errorf("error looking up app: %w", err) } diff --git a/cmd/spoon/download.go b/cmd/spoon/download.go new file mode 100644 index 0000000..7ae9192 --- /dev/null +++ b/cmd/spoon/download.go @@ -0,0 +1,103 @@ +package main + +import ( + "errors" + "fmt" + "os" + "path/filepath" + + "github.com/Bios-Marcel/spoon/pkg/scoop" + "github.com/spf13/cobra" +) + +func downloadCmd() *cobra.Command { + cmd := &cobra.Command{ + Use: "download", + Short: "Download all files required for a package", + Args: cobra.MinimumNArgs(1), + ValidArgsFunction: autocompleteAvailable, + RunE: RunE(func(cmd *cobra.Command, args []string) error { + arch := scoop.ArchitectureKey(must(cmd.Flags().GetString("arch"))) + force := must(cmd.Flags().GetBool("force")) + noHashCheck := must(cmd.Flags().GetBool("no-hash-check")) + + defaultScoop, err := scoop.NewScoop() + if err != nil { + return fmt.Errorf("error retrieving scoop instance: %w", err) + } + + for _, arg := range args { + app, err := defaultScoop.FindAvailableApp(arg) + if err != nil { + return fmt.Errorf("error looking up app: %w", err) + } + if app == nil { + return fmt.Errorf("app '%s' not found", arg) + } + + if err := app.LoadDetails( + scoop.DetailFieldArchitecture, + scoop.DetailFieldUrl, + scoop.DetailFieldHash, + ); err != nil { + return fmt.Errorf("error loading app details: %w", err) + } + + resolvedApp := app.ForArch(arch) + resultChan, err := resolvedApp.Download( + defaultScoop.CacheDir(), arch, !noHashCheck, force, + ) + if err != nil { + return err + } + + for result := range resultChan { + switch result := result.(type) { + case *scoop.CacheHit: + name := filepath.Base(result.Downloadable.URL) + fmt.Printf("Cache hit for '%s'\n", name) + case *scoop.FinishedDownload: + name := filepath.Base(result.Downloadable.URL) + fmt.Printf("Downloaded '%s'\n", name) + case error: + var checksumErr *scoop.ChecksumMismatchError + if errors.As(result, &checksumErr) { + fmt.Printf( + "Checksum mismatch:\n\rFile: '%s'\n\tExpected: '%s'\n\tActual: '%s'\n", + checksumErr.File, + checksumErr.Expected, + checksumErr.Actual, + ) + + // FIXME Find a better way to do this via + // returnvalue? + os.Exit(1) + } + if result != nil { + return result + } + } + } + } + + return nil + }), + } + + cmd.Flags().BoolP("force", "f", false, "Force download (overwrite cache)") + // FIXME No shorthand for now, since --h is help and seems to clash. + cmd.Flags().Bool("no-hash-check", false, "Skip hash verification (use with caution!)") + // We default to our system architecture here. If scoop encounters an + // unsupported arch, it is ignored. We'll do the same. + cmd.Flags().StringP("arch", "a", string(SystemArchitecture), + "use specified architecture, if app supports it") + cmd.RegisterFlagCompletionFunc("arch", cobra.FixedCompletions( + []string{ + string(scoop.ArchitectureKey32Bit), + string(scoop.ArchitectureKey64Bit), + string(scoop.ArchitectureKeyARM64), + }, + cobra.ShellCompDirectiveDefault)) + + return cmd +} diff --git a/cmd/spoon/install.go b/cmd/spoon/install.go index f7cb251..69d8c27 100644 --- a/cmd/spoon/install.go +++ b/cmd/spoon/install.go @@ -14,23 +14,34 @@ func installCmd() *cobra.Command { Short: "Install a package", Args: cobra.MinimumNArgs(1), ValidArgsFunction: autocompleteAvailable, - Run: func(cmd *cobra.Command, args []string) { - flags, err := getFlags(cmd, "global", "independent", "no-cache", "no-update-scoop", "skip", "arch") + RunE: RunE(func(cmd *cobra.Command, args []string) error { + // Flags we currently do not support + if must(cmd.Flags().GetBool("global")) { + flags, err := getFlags(cmd, "global", "independent", "no-cache", + "no-update-scoop", "skip", "arch") + if err != nil { + return err + } + os.Exit(execScoopCommand("install", append(flags, args...)...)) + } + + arch := must(cmd.Flags().GetString("arch")) + + defaultScoop, err := scoop.NewScoop() if err != nil { - fmt.Println(err) - os.Exit(1) + return fmt.Errorf("error retrieving scoop instance: %w", err) } - // Default path, where we can't do our simple optimisation of - // parallelising install and download, as we only have one package. - if len(args) == 1 { - os.Exit(execScoopCommand("install", append(flags, args...)...)) - return + installErrors := defaultScoop.InstallAll(args, scoop.ArchitectureKey(arch)) + for _, err := range installErrors { + fmt.Println(err) } - // FIXME Parallelise. - os.Exit(execScoopCommand("install", append(flags, args...)...)) - }, + if len(installErrors) > 0 { + os.Exit(1) + } + return nil + }), } cmd.Flags().BoolP("global", "g", false, "Install an app globally") diff --git a/cmd/spoon/main.go b/cmd/spoon/main.go index cfb381b..02e81ef 100644 --- a/cmd/spoon/main.go +++ b/cmd/spoon/main.go @@ -34,6 +34,7 @@ func main() { } rootCmd.AddCommand(searchCmd()) + rootCmd.AddCommand(downloadCmd()) rootCmd.AddCommand(installCmd()) rootCmd.AddCommand(uninstallCmd()) rootCmd.AddCommand(updateCmd()) diff --git a/cmd/spoon/search.go b/cmd/spoon/search.go index 6936d15..364ef7c 100644 --- a/cmd/spoon/search.go +++ b/cmd/spoon/search.go @@ -10,8 +10,6 @@ import ( "strings" "sync" - _ "runtime/pprof" - "github.com/Bios-Marcel/spoon/internal/cli" "github.com/Bios-Marcel/spoon/pkg/scoop" jsoniter "github.com/json-iterator/go" diff --git a/cmd/spoon/shell.go b/cmd/spoon/shell.go index 148afe7..7822ae4 100644 --- a/cmd/spoon/shell.go +++ b/cmd/spoon/shell.go @@ -176,9 +176,9 @@ func shellCmd() *cobra.Command { } if err := windows.CreateJunctions([][2]string{ - {defaultScoop.GetCacheDir(), tempScoop.GetCacheDir()}, - {defaultScoop.GetScoopInstallationDir(), tempScoop.GetScoopInstallationDir()}, - {defaultScoop.GetBucketsDir(), tempScoop.GetBucketsDir()}, + {defaultScoop.CacheDir(), tempScoop.CacheDir()}, + {defaultScoop.ScoopInstallationDir(), tempScoop.ScoopInstallationDir()}, + {defaultScoop.BucketDir(), tempScoop.BucketDir()}, }...); err != nil { return fmt.Errorf("error creating junctions: %w", err) } @@ -211,7 +211,7 @@ func shellCmd() *cobra.Command { // environment variables and some apps use env_add_path instead // of specifying shims. var app *scoop.InstalledApp - app, err = tempScoop.GetInstalledApp(dependency) + app, err = tempScoop.FindInstalledApp(dependency) if err != nil { break } diff --git a/cmd/spoon/uninstall.go b/cmd/spoon/uninstall.go index 3c9910f..b8ab439 100644 --- a/cmd/spoon/uninstall.go +++ b/cmd/spoon/uninstall.go @@ -25,28 +25,57 @@ func uninstallCmd() *cobra.Command { }, Args: cobra.MinimumNArgs(1), ValidArgsFunction: autocompleteInstalled, - Run: func(cmd *cobra.Command, args []string) { + RunE: RunE(func(cmd *cobra.Command, args []string) error { yes, err := cmd.Flags().GetBool("yes") if err != nil { - fmt.Println("error getting yes flag:", err) - os.Exit(1) + return fmt.Errorf("error getting yes flag: %w", err) } defaultScoop, err := scoop.NewScoop() if err != nil { - fmt.Println("error getting default scoop:", err) - os.Exit(1) + return fmt.Errorf("error getting default scoop: %w", err) } + if err := checkRunningProcesses(defaultScoop, args, yes); err != nil { - fmt.Println(err) + return fmt.Errorf("error checking running processes: %w", err) } - redirectedFlags, err := getFlags(cmd, "global", "purge") - if err != nil { - fmt.Println(err) - os.Exit(1) + // FIXME 3 funcs: FindInstalledApp, FindInstalledApps, + // InstalledApps. The later returns all of them, returning + // everything instead of finding something. + for _, arg := range args { + app, err := defaultScoop.FindInstalledApp(args[0]) + if err != nil { + return err + } + + // FIXME Is this good? What does scoop do? + if app == nil { + fmt.Printf("App '%s' is not intalled.\n", arg) + continue + } + + // FIXME We need to make the loading stuff less annoying. Can we + // have a special optimisation path / package so that we can + // still cover stuff such as search? + if err := app.LoadDetails(scoop.DetailFieldsAll...); err != nil { + return fmt.Errorf("error loading app details: %w", err) + } + + // FIXME This currently only uninstalls a specific version. We + // need multiple versions current, specific all? + if err := defaultScoop.Uninstall(app, app.Architecture); err != nil { + return fmt.Errorf("error uninstalling '%s': %w", arg, err) + } } - os.Exit(execScoopCommand("uninstall", append(redirectedFlags, args...)...)) - }, + + // redirectedFlags, err := getFlags(cmd, "global", "purge") + // if err != nil { + // fmt.Println(err) + // os.Exit(1) + // } + // os.Exit(execScoopCommand("uninstall", append(redirectedFlags, args...)...)) + return nil + }), } cmd.Flags().BoolP("global", "g", false, "Uninstall a globally installed app") @@ -65,7 +94,7 @@ func checkRunningProcesses(scoop *scoop.Scoop, args []string, yes bool) error { var processPrefixes []string for _, arg := range args { processPrefixes = append(processPrefixes, - strings.ToLower(filepath.Join(scoop.GetAppsDir(), arg)+"\\")) + strings.ToLower(filepath.Join(scoop.AppDir(), arg)+"\\")) } var processesToKill []shared.Process diff --git a/cmd/spoon/versions.go b/cmd/spoon/versions.go index 21ef625..5eecff6 100644 --- a/cmd/spoon/versions.go +++ b/cmd/spoon/versions.go @@ -19,7 +19,7 @@ func versionsCmd() *cobra.Command { return fmt.Errorf("error getting default scoop: %w", err) } - app, err := defaultScoop.GetAvailableApp(args[0]) + app, err := defaultScoop.FindAvailableApp(args[0]) if err != nil { return fmt.Errorf("error finding app: %w", err) } diff --git a/go.mod b/go.mod index 1536fcf..9b042f1 100644 --- a/go.mod +++ b/go.mod @@ -1,9 +1,10 @@ module github.com/Bios-Marcel/spoon -go 1.21.1 +go 1.22.0 require ( github.com/Bios-Marcel/versioncmp v0.0.0-20240329201707-2bd36cfebbc9 + github.com/cavaliergopher/grab/v3 v3.0.1 github.com/fatih/color v1.16.0 github.com/go-git/go-git/v5 v5.11.0 github.com/iamacarpet/go-win64api v0.0.0-20230324134531-ef6dbdd6db97 @@ -39,6 +40,7 @@ require ( github.com/modern-go/reflect2 v1.0.2 // indirect github.com/pjbgf/sha1cd v0.3.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/rivo/uniseg v0.4.4 // indirect github.com/scjalliance/comshim v0.0.0-20190308082608-cf06d2532c4e // indirect github.com/sergi/go-diff v1.1.0 // indirect github.com/skeema/knownhosts v1.2.1 // indirect @@ -47,7 +49,7 @@ require ( golang.org/x/crypto v0.16.0 // indirect golang.org/x/mod v0.12.0 // indirect golang.org/x/net v0.19.0 // indirect - golang.org/x/sys v0.15.0 // indirect + golang.org/x/sys v0.16.0 // indirect golang.org/x/tools v0.13.0 // indirect gopkg.in/warnings.v0 v0.1.2 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect diff --git a/go.sum b/go.sum index 6e7ec9c..f7378e4 100644 --- a/go.sum +++ b/go.sum @@ -18,6 +18,8 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPd github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0= github.com/capnspacehook/taskmaster v0.0.0-20210519235353-1629df7c85e9/go.mod h1:257CYs3Wd/CTlLQ3c72jKv+fFE2MV3WPNnV5jiroYUU= +github.com/cavaliergopher/grab/v3 v3.0.1 h1:4z7TkBfmPjmLAAmkkAZNX/6QJ1nNFdv3SdIHXju0Fr4= +github.com/cavaliergopher/grab/v3 v3.0.1/go.mod h1:1U/KNnD+Ft6JJiYoYBAimKH2XrYptb8Kl3DFGmsjpq4= github.com/cloudflare/circl v1.3.3 h1:fE/Qz0QdIGqeWfnwq0RE0R7MI51s0M2E4Ga9kq5AEMs= github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA= github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= @@ -125,8 +127,9 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rickb777/date v1.14.2/go.mod h1:swmf05C+hN+m8/Xh7gEq3uB6QJDNc5pQBWojKdHetOs= github.com/rickb777/plural v1.2.2/go.mod h1:xyHbelv4YvJE51gjMnHvk+U2e9zIysg6lTnSQK8XUYA= -github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= +github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis= +github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro= github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M= github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA= @@ -216,8 +219,8 @@ golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= -golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.16.0 h1:xWw16ngr6ZMtmxDyKyIgsE93KNKz5HKmMa3b8ALHidU= +golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc= diff --git a/internal/windows/env.go b/internal/windows/env.go index bd82618..a7790df 100644 --- a/internal/windows/env.go +++ b/internal/windows/env.go @@ -1,15 +1,95 @@ package windows import ( + "bytes" "encoding/json" "fmt" "os" "os/exec" + "slices" + "strings" ) +type Paths []string + +// ParsePath will break the path variable content down into separate paths. This +// also handles quoting. The order is preserved. +func ParsePath(value string) Paths { + // Technically we could also use strings.FieldFunc, but we got to manually + // cut of the quotes anyway then, so we'll just do the whole thing manually. + var values []string + var quoteOpen bool + var nextStart int + for index, char := range value { + if char == '"' { + if quoteOpen { + // +1 to skip the open quote + values = append(values, value[nextStart+1:index]) + // End quote means we'll have a separator next, so we start at + // the next path char. + nextStart = index + 2 + } + + quoteOpen = !quoteOpen + } else if char == ';' && index > nextStart { + if quoteOpen { + continue + } + + values = append(values, value[nextStart:index]) + nextStart = index + 1 + } + } + + // Last element if applicable, since the path could also end on a semicolon + // or quote. + if nextStart < len(value) { + values = append(values, value[nextStart:]) + } + + return Paths(values) +} + +// Remove returns a new path object that doesn't contain any of the specified +// paths. +func (p Paths) Remove(paths ...string) Paths { + p = slices.DeleteFunc(p, func(value string) bool { + // FIXME This should sanitize the path separators and such. We also need + // tests for this. + return slices.Contains(paths, value) + }) + return p +} + +// Preprend will create a new Paths object, adding the supplied paths infront, +// using the given order. +func (p Paths) Prepend(paths ...string) Paths { + newPath := make(Paths, 0, len(p)+len(paths)) + newPath = append(newPath, paths...) + newPath = append(newPath, p...) + return newPath +} + +// Creates a new path string, where all entries are quoted. +func (p Paths) String() string { + var buffer bytes.Buffer + for i := 0; i < len(p); i++ { + if i != 0 { + buffer.WriteRune(';') + } + + // FIXME Only quote if necessary? Only if contains semicolon? + buffer.WriteRune('"') + buffer.WriteString(p[i]) + buffer.WriteRune('"') + } + return buffer.String() +} + func GetPersistentEnvValues() (map[string]string, error) { cmd := exec.Command( "powershell", + "-NoLogo", "-NoProfile", "[Environment]::GetEnvironmentVariables('User') | ConvertTo-Json", ) @@ -36,12 +116,34 @@ func GetPersistentEnvValues() (map[string]string, error) { return result, nil } +// GetPersistentEnvValue retrieves a persistent user level environment variable. +// The first returned value is the key and the second the value. While the key +// is defined in the query, the casing might be different, which COULD matter. +// If nothing was found, we return empty strings without an error. +func GetPersistentEnvValue(key string) (string, string, error) { + // While we could directly call GetEnvironmentVariable, we want to find out + // he string, therefore we use the result of the GetAll call. + + allVars, err := GetPersistentEnvValues() + if err != nil { + return "", "", fmt.Errorf("error retrieving variables: %w", err) + } + + for keyPersisted, val := range allVars { + if strings.EqualFold(key, keyPersisted) { + return keyPersisted, val, nil + } + } + return "", "", nil +} + // Sets a User-Level Environment variable. An empty value will remove the key // completly. func SetPersistentEnvValue(key, value string) error { cmd := exec.Command( "powershell", "-NoProfile", + "-NoLogo", "-Command", "[Environment]::SetEnvironmentVariable('"+key+"','"+value+"','User')", ) @@ -50,3 +152,30 @@ func SetPersistentEnvValue(key, value string) error { cmd.Stdin = os.Stdin return cmd.Run() } + +func SetPersistentEnvValues(vars ...[2]string) error { + if len(vars) == 0 { + return nil + } + + var command bytes.Buffer + for _, pair := range vars { + command.WriteString("[Environment]::SetEnvironmentVariable('") + command.WriteString(pair[0]) + command.WriteString("','") + command.WriteString(pair[1]) + command.WriteString("','User');") + } + + cmd := exec.Command( + "powershell", + "-NoProfile", + "-NoLogo", + "-Command", + command.String(), + ) + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + cmd.Stdin = os.Stdin + return cmd.Run() +} diff --git a/internal/windows/env_test.go b/internal/windows/env_test.go new file mode 100644 index 0000000..116b89b --- /dev/null +++ b/internal/windows/env_test.go @@ -0,0 +1,14 @@ +package windows_test + +import ( + "testing" + + "github.com/Bios-Marcel/spoon/internal/windows" + "github.com/stretchr/testify/require" +) + +func Test_ParsePath(t *testing.T) { + path := windows.ParsePath(`C:\path_a;"C:\path_b";"C:\path_;";C:\path_c`) + require.Equal(t, []string{`C:\path_a`, `C:\path_b`, `C:\path_;`, `C:\path_c`}, []string(path)) + require.Equal(t, `"C:\path_a";"C:\path_b";"C:\path_;";"C:\path_c"`, path.String()) +} diff --git a/internal/windows/windows.go b/internal/windows/windows.go index f10f6d9..e0064ed 100644 --- a/internal/windows/windows.go +++ b/internal/windows/windows.go @@ -67,8 +67,10 @@ func GetShellExecutable() (string, error) { } // Depending on whether we are shimmed or not, our parent might be - // a shim, so we'll try ignoring this and going deeper. - if lowered := strings.ToLower(name); lowered == "spoon.exe" || lowered == "spoon" { + // a shim, so we'll try ignoring this and going deeper. We'll + // additionally ignore go.exe, as this helps during dev, using `go run`. + if lowered := strings.ToLower(name); lowered == "spoon.exe" || + lowered == "spoon" || lowered == "go" || lowered == "go.exe" { parentId = id continue } diff --git a/pkg/scoop/manifest.go b/pkg/scoop/manifest.go new file mode 100644 index 0000000..1ab3764 --- /dev/null +++ b/pkg/scoop/manifest.go @@ -0,0 +1,318 @@ +package scoop + +import ( + "fmt" + "io" + "os" + "slices" + "strings" + + jsoniter "github.com/json-iterator/go" +) + +const ( + DetailFieldBin = "bin" + DetailFieldShortcuts = "shortcuts" + DetailFieldUrl = "url" + DetailFieldHash = "hash" + DetailFieldArchitecture = "architecture" + DetailFieldDescription = "description" + DetailFieldVersion = "version" + DetailFieldNotes = "notes" + DetailFieldDepends = "depends" + DetailFieldEnvSet = "env_set" + DetailFieldEnvAddPath = "env_add_path" + DetailFieldExtractDir = "extract_dir" + DetailFieldExtractTo = "extract_to" + DetailFieldPostInstall = "post_install" + DetailFieldPreInstall = "pre_install" + DetailFieldPreUninstall = "pre_uninstall" + DetailFieldPostUninstall = "post_uninstall" + DetailFieldInstaller = "installer" + DetailFieldUninstaller = "uninstaller" + DetailFieldInnoSetup = "innosetup" +) + +// DetailFieldsAll is a list of all available DetailFields to load during +// [App.LoadDetails]. Use these if you need all fields or don't care whether +// unneeded fields are being loaded. +var DetailFieldsAll = []string{ + DetailFieldBin, + DetailFieldShortcuts, + DetailFieldUrl, + DetailFieldHash, + DetailFieldArchitecture, + DetailFieldDescription, + DetailFieldVersion, + DetailFieldNotes, + DetailFieldDepends, + DetailFieldEnvSet, + DetailFieldEnvAddPath, + DetailFieldExtractDir, + DetailFieldExtractTo, + DetailFieldPostInstall, + DetailFieldPreInstall, + DetailFieldPreUninstall, + DetailFieldPostUninstall, + DetailFieldInstaller, + DetailFieldUninstaller, + DetailFieldInnoSetup, +} + +// manifestIter gives you an iterator with a big enough size to read any +// manifest without reallocations. +func manifestIter() *jsoniter.Iterator { + return jsoniter.Parse(jsoniter.ConfigFastest, nil, 1024*128) +} + +// LoadDetails will load additional data regarding the manifest, such as +// description and version information. This causes IO on your drive and +// therefore isn't done by default. +func (a *App) LoadDetails(fields ...string) error { + return a.LoadDetailsWithIter(manifestIter(), fields...) +} + +// LoadDetails will load additional data regarding the manifest, such as +// description and version information. This causes IO on your drive and +// therefore isn't done by default. +func (a *App) LoadDetailsWithIter(iter *jsoniter.Iterator, fields ...string) error { + file, err := os.Open(a.manifestPath) + if err != nil { + return fmt.Errorf("error opening manifest: %w", err) + } + defer file.Close() + + return a.loadDetailFromManifestWithIter(iter, file, fields...) +} + +func mergeIntoDownloadables(urls, hashes, extractDirs, extractTos []string) []Downloadable { + // It can happen that we have different extract_dirs, but only one archive, + // containing both architectures. This should also never be empty, but at + // least of size one, so we'll never allocate for naught. + downloadables := make([]Downloadable, max(len(urls), len(extractDirs), len(extractTos))) + + // We assume that we have the same length in each. While this + // hasn't been specified in the app manifests wiki page, it's + // the seemingly only sensible thing to me. + // If we are missing extract_dir or extract_to entries, it's fine, as we use + // nonpointer values anyway and simple default to empty, which means + // application directory. + for index, value := range urls { + downloadables[index].URL = value + } + for index, value := range hashes { + downloadables[index].Hash = value + } + for index, value := range extractDirs { + downloadables[index].ExtractDir = value + } + for index, value := range extractTos { + downloadables[index].ExtractTo = value + } + + return downloadables +} + +// LoadDetails will load additional data regarding the manifest, such as +// description and version information. This causes IO on your drive and +// therefore isn't done by default. +func (a *App) loadDetailFromManifestWithIter( + iter *jsoniter.Iterator, + manifest io.Reader, + fields ...string, +) error { + iter.Reset(manifest) + + var urls, hashes, extractDirs, extractTos []string + for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { + if !slices.Contains(fields, field) { + iter.Skip() + continue + } + + switch field { + case DetailFieldDescription: + a.Description = iter.ReadString() + case DetailFieldVersion: + a.Version = iter.ReadString() + case DetailFieldUrl: + urls = parseStringOrArray(iter) + case DetailFieldHash: + hashes = parseStringOrArray(iter) + case DetailFieldShortcuts: + a.Shortcuts = parseBin(iter) + case DetailFieldBin: + a.Bin = parseBin(iter) + case DetailFieldArchitecture: + // Preallocate to 3, as we support at max 3 architectures + a.Architecture = make(map[ArchitectureKey]*Architecture, 3) + for arch := iter.ReadObject(); arch != ""; arch = iter.ReadObject() { + var archValue Architecture + a.Architecture[ArchitectureKey(arch)] = &archValue + + var urls, hashes, extractDirs []string + for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { + switch field { + case "url": + urls = parseStringOrArray(iter) + case "hash": + hashes = parseStringOrArray(iter) + case "extract_dir": + extractDirs = parseStringOrArray(iter) + case "bin": + archValue.Bin = parseBin(iter) + case "shortcuts": + archValue.Shortcuts = parseBin(iter) + case "installer": + installer := parseInstaller(iter) + archValue.Installer = &installer + case "uninstaller": + uninstaller := Uninstaller(parseInstaller(iter)) + archValue.Uninstaller = &uninstaller + default: + iter.Skip() + } + } + + // extract_to is always on the root level, so we pass nil + archValue.Downloadables = mergeIntoDownloadables(urls, hashes, extractDirs, nil) + } + case DetailFieldDepends: + // Array at top level to create multiple entries + if iter.WhatIsNext() == jsoniter.ArrayValue { + for iter.ReadArray() { + a.Depends = append(a.Depends, a.parseDependency(iter.ReadString())) + } + } else { + a.Depends = []Dependency{a.parseDependency(iter.ReadString())} + } + case DetailFieldEnvAddPath: + a.EnvAddPath = parseStringOrArray(iter) + case DetailFieldEnvSet: + for key := iter.ReadObject(); key != ""; key = iter.ReadObject() { + a.EnvSet = append(a.EnvSet, EnvVar{Key: key, Value: iter.ReadString()}) + } + case DetailFieldInstaller: + installer := parseInstaller(iter) + a.Installer = &installer + case DetailFieldUninstaller: + uninstaller := Uninstaller(parseInstaller(iter)) + a.Uninstaller = &uninstaller + case DetailFieldInnoSetup: + a.InnoSetup = iter.ReadBool() + case DetailFieldPreInstall: + a.PreInstall = parseStringOrArray(iter) + case DetailFieldPostInstall: + a.PostInstall = parseStringOrArray(iter) + case DetailFieldPreUninstall: + a.PreUninstall = parseStringOrArray(iter) + case DetailFieldPostUninstall: + a.PostUninstall = parseStringOrArray(iter) + case DetailFieldExtractDir: + extractDirs = parseStringOrArray(iter) + case DetailFieldExtractTo: + extractTos = parseStringOrArray(iter) + case DetailFieldNotes: + if iter.WhatIsNext() == jsoniter.ArrayValue { + var lines []string + for iter.ReadArray() { + lines = append(lines, iter.ReadString()) + } + a.Notes = strings.Join(lines, "\n") + } else { + a.Notes = iter.ReadString() + } + default: + iter.Skip() + } + } + + if iter.Error != nil { + return fmt.Errorf("error parsing json: %w", iter.Error) + } + + // If there are no URLs at the root level, that means they are in the + // arch-specific instructions. In this case, we'll only access the + // ExtractTo / ExtractDir when resolving a certain arch. + if len(urls) > 0 { + a.Downloadables = mergeIntoDownloadables(urls, hashes, extractDirs, extractTos) + } + + return nil +} + +func parseInstaller(iter *jsoniter.Iterator) Installer { + installer := Installer{} + for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { + switch field { + case "file": + installer.File = iter.ReadString() + case "script": + installer.Script = parseStringOrArray(iter) + case "args": + installer.Args = parseStringOrArray(iter) + case "keep": + installer.Keep = iter.ReadBool() + default: + iter.Skip() + } + } + return installer +} + +func parseBin(iter *jsoniter.Iterator) []Bin { + // Array at top level to create multiple entries + if iter.WhatIsNext() == jsoniter.ArrayValue { + var bins []Bin + for iter.ReadArray() { + // There are nested arrays, for shim creation, with format: + // binary alias [args...] + if iter.WhatIsNext() == jsoniter.ArrayValue { + var bin Bin + if iter.ReadArray() { + bin.Name = iter.ReadString() + } + if iter.ReadArray() { + bin.Alias = iter.ReadString() + } + for iter.ReadArray() { + bin.Args = append(bin.Args, iter.ReadString()) + } + bins = append(bins, bin) + } else { + // String in the root level array to add to path + bins = append(bins, Bin{Name: iter.ReadString()}) + } + } + return bins + } + + // String value at root level to add to path. + return []Bin{{Name: iter.ReadString()}} +} + +func parseStringOrArray(iter *jsoniter.Iterator) []string { + if iter.WhatIsNext() == jsoniter.ArrayValue { + var val []string + for iter.ReadArray() { + val = append(val, iter.ReadString()) + } + return val + } + + return []string{iter.ReadString()} +} + +func (a App) parseDependency(value string) Dependency { + parts := strings.SplitN(value, "/", 1) + switch len(parts) { + case 0: + // Should be a broken manifest + return Dependency{} + case 1: + // No bucket means same bucket. + return Dependency{Bucket: a.Bucket.Name(), Name: parts[0]} + default: + return Dependency{Bucket: parts[0], Name: parts[1]} + } +} diff --git a/pkg/scoop/scoop.go b/pkg/scoop/scoop.go index 6295e9b..870ce3b 100644 --- a/pkg/scoop/scoop.go +++ b/pkg/scoop/scoop.go @@ -1,22 +1,31 @@ package scoop import ( + "archive/zip" "bytes" "context" + "crypto/md5" + "crypto/sha1" + "crypto/sha256" + "crypto/sha512" + "encoding/hex" "encoding/json" "errors" "fmt" + "hash" "io" "math" "os" + "os/exec" "path/filepath" "regexp" - "slices" + "strconv" "strings" "github.com/Bios-Marcel/spoon/internal/git" "github.com/Bios-Marcel/spoon/internal/windows" "github.com/Bios-Marcel/versioncmp" + "github.com/cavaliergopher/grab/v3" jsoniter "github.com/json-iterator/go" ) @@ -61,7 +70,7 @@ func (b *Bucket) ManifestDir() string { return b.manifestDir } -func (b *Bucket) GetApp(name string) *App { +func (b *Bucket) FindApp(name string) *App { potentialManifest := filepath.Join(b.ManifestDir(), name+".json") if _, err := os.Stat(potentialManifest); err == nil { return &App{ @@ -103,13 +112,13 @@ var ErrBucketNotFound = errors.New("bucket not found") // GetBucket constructs a new bucket object pointing at the given bucket. At // this point, the bucket might not necessarily exist. func (scoop *Scoop) GetBucket(name string) *Bucket { - return &Bucket{rootDir: filepath.Join(scoop.GetBucketsDir(), name)} + return &Bucket{rootDir: filepath.Join(scoop.BucketDir(), name)} } -func (scoop *Scoop) GetAvailableApp(name string) (*App, error) { +func (scoop *Scoop) FindAvailableApp(name string) (*App, error) { bucket, name, _ := ParseAppIdentifier(name) if bucket != "" { - return scoop.GetBucket(bucket).GetApp(name), nil + return scoop.GetBucket(bucket).FindApp(name), nil } buckets, err := scoop.GetLocalBuckets() @@ -117,23 +126,23 @@ func (scoop *Scoop) GetAvailableApp(name string) (*App, error) { return nil, fmt.Errorf("error getting local buckets: %w", err) } for _, bucket := range buckets { - if app := bucket.GetApp(name); app != nil { + if app := bucket.FindApp(name); app != nil { return app, nil } } return nil, nil } -func (scoop *Scoop) GetInstalledApp(name string) (*InstalledApp, error) { +func (scoop *Scoop) FindInstalledApp(name string) (*InstalledApp, error) { iter := jsoniter.Parse(jsoniter.ConfigFastest, nil, 256) - return scoop.getInstalledApp(iter, name) + return scoop.findInstalledApp(iter, name) } -func (scoop *Scoop) getInstalledApp(iter *jsoniter.Iterator, name string) (*InstalledApp, error) { +func (scoop *Scoop) findInstalledApp(iter *jsoniter.Iterator, name string) (*InstalledApp, error) { _, name, _ = ParseAppIdentifier(name) name = strings.ToLower(name) - appDir := filepath.Join(scoop.GetAppsDir(), name, "current") + appDir := filepath.Join(scoop.AppDir(), name, "current") installJson, err := os.Open(filepath.Join(appDir, "install.json")) if err != nil { @@ -147,11 +156,14 @@ func (scoop *Scoop) getInstalledApp(iter *jsoniter.Iterator, name string) (*Inst iter.Reset(installJson) var ( - bucketName string - hold bool + bucketName string + architecture string + hold bool ) for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { switch field { + case "architecture": + architecture = iter.ReadString() case "bucket": bucketName = iter.ReadString() case "hold": @@ -167,7 +179,8 @@ func (scoop *Scoop) getInstalledApp(iter *jsoniter.Iterator, name string) (*Inst } return &InstalledApp{ - Hold: hold, + Hold: hold, + Architecture: ArchitectureKey(architecture), App: &App{ Bucket: bucket, Name: name, @@ -213,7 +226,7 @@ type KnownBucket struct { // GetKnownBuckets returns the list of available "default" buckets that are // available, but might have not necessarily been installed locally. func (scoop *Scoop) GetKnownBuckets() ([]KnownBucket, error) { - file, err := os.Open(filepath.Join(scoop.GetScoopInstallationDir(), "buckets.json")) + file, err := os.Open(filepath.Join(scoop.ScoopInstallationDir(), "buckets.json")) if err != nil { return nil, fmt.Errorf("error opening buckets.json: %w", err) } @@ -234,7 +247,7 @@ func (scoop *Scoop) GetKnownBuckets() ([]KnownBucket, error) { // GetLocalBuckets is an API representation of locally installed buckets. func (scoop *Scoop) GetLocalBuckets() ([]*Bucket, error) { - potentialBuckets, err := windows.GetDirFilenames(scoop.GetBucketsDir()) + potentialBuckets, err := windows.GetDirFilenames(scoop.BucketDir()) if err != nil { return nil, fmt.Errorf("error reading bucket names: %w", err) } @@ -243,7 +256,7 @@ func (scoop *Scoop) GetLocalBuckets() ([]*Bucket, error) { for _, potentialBucket := range potentialBuckets { // While the bucket folder SHOULD only contain buckets, one could // accidentally place ANYTHING else in it, even textfiles. - absBucketPath := filepath.Join(scoop.GetBucketsDir(), potentialBucket) + absBucketPath := filepath.Join(scoop.BucketDir(), potentialBucket) file, err := os.Stat(absBucketPath) if err != nil { return nil, fmt.Errorf("error stat-ing potential bucket: %w", err) @@ -261,6 +274,9 @@ func (scoop *Scoop) GetLocalBuckets() ([]*Bucket, error) { // may not be part of a bucket. "Headless" manifests are also a thing, for // example when you are using an auto-generated manifest for a version that's // not available anymore. In that case, scoop will lose the bucket information. +// +// Note that this structure doesn't reflect the same schema as the scoop +// manifests, as we are trying to make usage easier, not just as hard. type App struct { Name string `json:"name"` Description string `json:"description"` @@ -272,17 +288,21 @@ type App struct { EnvAddPath []string `json:"env_add_path"` EnvSet []EnvVar `json:"env_set"` + Downloadables []Downloadable + Depends []Dependency `json:"depends"` - URL []string `json:"url"` Architecture map[ArchitectureKey]*Architecture `json:"architecture"` InnoSetup bool `json:"innosetup"` - Installer *Installer `json:"installer"` - PreInstall []string `json:"pre_install"` - PostInstall []string `json:"post_install"` - ExtractTo []string `json:"extract_to"` - // ExtractDir specifies which dir should be extracted from the downloaded - // archive. However, there might be more URLs than there are URLs. - ExtractDir []string `json:"extract_dir"` + // Installer deprecates msi + Installer *Installer `json:"installer"` + Uninstaller *Uninstaller `json:"uninstaller"` + PreInstall []string `json:"pre_install"` + PostInstall []string `json:"post_install"` + PreUninstall []string `json:"pre_uninstall"` + PostUninstall []string `json:"post_uninstall"` + ExtractTo []string `json:"extract_to"` + + // Spoon "internals" Bucket *Bucket `json:"-"` manifestPath string @@ -293,6 +313,9 @@ type InstalledApp struct { // Hold indicates whether the app should be kept on the currently installed // version. It's versioning pinning. Hold bool + // Archictecture defines which architecture was used for installation. On a + // 64Bit system for example, this could also be 32Bit, but not vice versa. + Architecture ArchitectureKey } type OutdatedApp struct { @@ -329,13 +352,14 @@ const ( ) type Architecture struct { - Items []ArchitectureItem `json:"items"` + Downloadables []Downloadable `json:"items"` Bin []Bin Shortcuts []Bin // Installer replaces MSI - Installer Installer + Installer *Installer + Uninstaller *Uninstaller // PreInstall contains a list of commands to execute before installation. // Note that PreUninstall isn't supported in ArchitectureItem, even though @@ -347,251 +371,72 @@ type Architecture struct { PostInstall []string } -type ArchitectureItem struct { - URL string - Hash string +type Downloadable struct { + URL string + Hash string + // ExtractDir specifies which dir should be extracted from the downloaded + // archive. However, there might be more URLs than there are ExtractDirs. ExtractDir string + ExtractTo string } type Installer struct { // File is the installer executable. If not specified, this will - // autoamtically be set to the last item of the URLs. + // automatically be set to the last item of the URLs. Note, that this will + // be looked up in the extracted dirs, if explicitly specified. File string Script []string Args []string Keep bool } -func (a App) ManifestPath() string { - return a.manifestPath -} - -const ( - DetailFieldBin = "bin" - DetailFieldShortcuts = "shortcuts" - DetailFieldUrl = "url" - DetailFieldArchitecture = "architecture" - DetailFieldDescription = "description" - DetailFieldVersion = "version" - DetailFieldNotes = "notes" - DetailFieldDepends = "depends" - DetailFieldEnvSet = "env_set" - DetailFieldEnvAddPath = "env_add_path" - DetailFieldExtractDir = "extract_dir" - DetailFieldExtractTo = "extract_to" - DetailFieldPostInstall = "post_install" - DetailFieldPreInstall = "pre_install" - DetailFieldInstaller = "installer" - DetailFieldInnoSetup = "innosetup" -) - -// LoadDetails will load additional data regarding the manifest, such as -// description and version information. This causes IO on your drive and -// therefore isn't done by default. -func (a *App) LoadDetails(fields ...string) error { - iter := jsoniter.Parse(jsoniter.ConfigFastest, nil, 1024*128) - return a.LoadDetailsWithIter(iter, fields...) -} - -// LoadDetails will load additional data regarding the manifest, such as -// description and version information. This causes IO on your drive and -// therefore isn't done by default. -func (a *App) LoadDetailsWithIter(iter *jsoniter.Iterator, fields ...string) error { - file, err := os.Open(a.manifestPath) - if err != nil { - return fmt.Errorf("error opening manifest: %w", err) - } - defer file.Close() - - iter.Reset(file) - - for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { - if !slices.Contains(fields, field) { - iter.Skip() - continue - } - - switch field { - case DetailFieldDescription: - a.Description = iter.ReadString() - case DetailFieldVersion: - a.Version = iter.ReadString() - case DetailFieldUrl: - a.URL = parseStringOrArray(iter) - case DetailFieldShortcuts: - a.Shortcuts = parseBin(iter) - case DetailFieldBin: - a.Bin = parseBin(iter) - case DetailFieldArchitecture: - // Preallocate to 3, as we support at max 3 architectures - a.Architecture = make(map[ArchitectureKey]*Architecture, 3) - for arch := iter.ReadObject(); arch != ""; arch = iter.ReadObject() { - var archValue Architecture - a.Architecture[ArchitectureKey(arch)] = &archValue - - var urls, hashes, extractDirs []string - for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { - switch field { - case "url": - urls = parseStringOrArray(iter) - case "hash": - hashes = parseStringOrArray(iter) - case "extract_dir": - extractDirs = parseStringOrArray(iter) - case "bin": - archValue.Bin = parseBin(iter) - case "shortcuts": - archValue.Shortcuts = parseBin(iter) - default: - iter.Skip() - } - } - - // We use non-pointers, as we'll have everything initiliased - // already then. It can happen that we have different - // extract_dirs, but only one archive, containing both - // architectures. - archValue.Items = make([]ArchitectureItem, max(len(urls), len(extractDirs))) - - // We assume that we have the same length in each. While this - // hasn't been specified in the app manifests wiki page, it's - // the seemingly only sensible thing to me. - for index, value := range urls { - archValue.Items[index].URL = value - } - for index, value := range hashes { - archValue.Items[index].Hash = value - } - for index, value := range extractDirs { - archValue.Items[index].ExtractDir = value - } - } - case DetailFieldDepends: - // Array at top level to create multiple entries - if iter.WhatIsNext() == jsoniter.ArrayValue { - for iter.ReadArray() { - a.Depends = append(a.Depends, a.parseDependency(iter.ReadString())) - } - } else { - a.Depends = []Dependency{a.parseDependency(iter.ReadString())} - } - case DetailFieldEnvAddPath: - a.EnvAddPath = parseStringOrArray(iter) - case DetailFieldEnvSet: - for key := iter.ReadObject(); key != ""; key = iter.ReadObject() { - a.EnvSet = append(a.EnvSet, EnvVar{Key: key, Value: iter.ReadString()}) - } - case DetailFieldInstaller: - a.Installer = &Installer{} - for field := iter.ReadObject(); field != ""; field = iter.ReadObject() { - switch field { - case "file": - a.Installer.File = iter.ReadString() - case "script": - a.Installer.Script = parseStringOrArray(iter) - case "args": - a.Installer.Args = parseStringOrArray(iter) - case "keep": - a.Installer.Keep = iter.ReadBool() - default: - iter.Skip() - } - } - case DetailFieldInnoSetup: - a.InnoSetup = iter.ReadBool() - case DetailFieldPreInstall: - a.PreInstall = parseStringOrArray(iter) - case DetailFieldPostInstall: - a.PostInstall = parseStringOrArray(iter) - case DetailFieldExtractDir: - a.ExtractDir = parseStringOrArray(iter) - case DetailFieldExtractTo: - a.ExtractTo = parseStringOrArray(iter) - case DetailFieldNotes: - if iter.WhatIsNext() == jsoniter.ArrayValue { - var lines []string - for iter.ReadArray() { - lines = append(lines, iter.ReadString()) - } - a.Notes = strings.Join(lines, "\n") - } else { - a.Notes = iter.ReadString() - } - default: - iter.Skip() +type Uninstaller Installer + +// invoke will run the installer script or file. This method is implemented on a +// non-pointer as we manipulate the script. +func (installer Installer) invoke(scoop *Scoop, dir string, arch ArchitectureKey) error { + // File and Script are mutually exclusive and Keep is only used if script is + // not set. However, we automatically set file to the last downloaded file + // if none is set, we then pass this to the script if any is present. + if len(installer.Script) > 0 { + variableSubstitutions := map[string]string{ + "$fname": installer.File, + "$dir": dir, + "$architecture": string(arch), + // FIXME We don't intend to support writing back the manifest into + // our context for now, as it seems only 1 or 2 apps actually do + // this. Instead, we should try to prepend a line that parses the + // manifest inline and creates the variable locally. + "$manifest": "TODO", } - } - - if iter.Error != nil { - return fmt.Errorf("error parsing json: %w", iter.Error) - } - - if a.Installer != nil { - if len(a.Architecture) > 0 { - // FIXME Get Architecvhture - } else if len(a.URL) > 0 { - a.Installer.File = a.URL[len(a.URL)-1] + for index, line := range installer.Script { + installer.Script[index] = substituteVariables(line, variableSubstitutions) } - } - - return nil -} - -func parseBin(iter *jsoniter.Iterator) []Bin { - // Array at top level to create multiple entries - if iter.WhatIsNext() == jsoniter.ArrayValue { - var bins []Bin - for iter.ReadArray() { - // There are nested arrays, for shim creation, with format: - // binary alias [args...] - if iter.WhatIsNext() == jsoniter.ArrayValue { - var bin Bin - if iter.ReadArray() { - bin.Name = iter.ReadString() - } - if iter.ReadArray() { - bin.Alias = iter.ReadString() - } - for iter.ReadArray() { - bin.Args = append(bin.Args, iter.ReadString()) - } - bins = append(bins, bin) - } else { - // String in the root level array to add to path - bins = append(bins, Bin{Name: iter.ReadString()}) - } + if err := scoop.runScript(installer.Script); err != nil { + return fmt.Errorf("error running installer: %w", err) } - return bins - } - - // String value at root level to add to path. - return []Bin{{Name: iter.ReadString()}} -} - -func parseStringOrArray(iter *jsoniter.Iterator) []string { - if iter.WhatIsNext() == jsoniter.ArrayValue { - var val []string - for iter.ReadArray() { - val = append(val, iter.ReadString()) + } else if installer.File != "" { + // FIXME RUN! Not extract? + + if !installer.Keep { + // FIXME Okay ... it seems scoop downloads the files not only into + // cache, but also into the installation directory. This seems a bit + // wasteful to me. Instead, we should copy the files into the dir + // only if we actually want to keep them. This way we can prevent + // useless copy and remove actions. + // + // This implementation shouldn't be part of the download, but + // instead be done during installation, manually checking both + // uninstaller.keep and installer.keep, copying if necessary and + // correctly invoking with the resulting paths. } - return val } - return []string{iter.ReadString()} + return nil } -func (a App) parseDependency(value string) Dependency { - parts := strings.SplitN(value, "/", 1) - switch len(parts) { - case 0: - // Should be a broken manifest - return Dependency{} - case 1: - // No bucket means same bucket. - return Dependency{Bucket: a.Bucket.Name(), Name: parts[0]} - default: - return Dependency{Bucket: parts[0], Name: parts[1]} - } +func (a *App) ManifestPath() string { + return a.manifestPath } type Dependencies struct { @@ -603,7 +448,7 @@ func (scoop *Scoop) DependencyTree(a *App) (*Dependencies, error) { dependencies := Dependencies{App: a} for _, dependency := range a.Depends { bucket := scoop.GetBucket(dependency.Bucket) - dependencyApp := bucket.GetApp(dependency.Name) + dependencyApp := bucket.FindApp(dependency.Name) subTree, err := scoop.DependencyTree(dependencyApp) if err != nil { return nil, fmt.Errorf("error getting sub dependency tree: %w", err) @@ -628,7 +473,7 @@ func (scoop *Scoop) ReverseDependencyTree(apps []*App, app *App) *Dependencies { } func (scoop *Scoop) GetOutdatedApps() ([]*OutdatedApp, error) { - installJSONPaths, err := filepath.Glob(filepath.Join(scoop.GetAppsDir(), "*/current/install.json")) + installJSONPaths, err := filepath.Glob(filepath.Join(scoop.AppDir(), "*/current/install.json")) if err != nil { return nil, fmt.Errorf("error globbing manifests: %w", err) } @@ -668,12 +513,12 @@ func (scoop *Scoop) GetOutdatedApps() ([]*OutdatedApp, error) { // We don't access the bucket directly, as this function supports // searching with and without bucket. - app, err := scoop.GetAvailableApp(appName) + app, err := scoop.FindAvailableApp(appName) if err != nil { return nil, fmt.Errorf("error getting app '%s' from bucket: %w", appName, err) } - installedApp, err := scoop.getInstalledApp(iter, appName) + installedApp, err := scoop.findInstalledApp(iter, appName) if err != nil { return nil, fmt.Errorf("error getting installed app '%s': %w", appName, err) } @@ -709,8 +554,8 @@ func (scoop *Scoop) GetOutdatedApps() ([]*OutdatedApp, error) { return outdated, nil } -func (scoop *Scoop) GetInstalledApps() ([]*InstalledApp, error) { - manifestPaths, err := filepath.Glob(filepath.Join(scoop.GetAppsDir(), "*/current/manifest.json")) +func (scoop *Scoop) InstalledApps() ([]*InstalledApp, error) { + manifestPaths, err := filepath.Glob(filepath.Join(scoop.AppDir(), "*/current/manifest.json")) if err != nil { return nil, fmt.Errorf("error globbing manifests: %w", err) } @@ -742,12 +587,16 @@ func (scoop *Scoop) GetInstalledApps() ([]*InstalledApp, error) { return apps, nil } -func (scoop *Scoop) GetBucketsDir() string { +func (scoop *Scoop) BucketDir() string { return filepath.Join(scoop.scoopRoot, "buckets") } -func (scoop *Scoop) GetScoopInstallationDir() string { - return filepath.Join(scoop.GetAppsDir(), "scoop", "current") +func (scoop *Scoop) PersistDir() string { + return filepath.Join(scoop.scoopRoot, "persist") +} + +func (scoop *Scoop) ScoopInstallationDir() string { + return filepath.Join(scoop.AppDir(), "scoop", "current") } func GetDefaultScoopDir() (string, error) { @@ -766,31 +615,880 @@ func GetDefaultScoopDir() (string, error) { return filepath.Join(home, "scoop"), nil } -func (scoop *Scoop) Install(apps []string, arch ArchitectureKey) error { - for _, inputName := range apps { - app, err := scoop.GetAvailableApp(inputName) +func (scoop *Scoop) runScript(lines []string) error { + if len(lines) == 0 { + return nil + } + + shell, err := windows.GetShellExecutable() + if err != nil { + // FIXME Does this need to be terminal? + return fmt.Errorf("error getting shell") + } + shell = strings.ToLower(shell) + switch shell { + case "pwsh.exe", "powershell.exe": + default: + return fmt.Errorf("shell '%s' not supported right now", shell) + } + + cmd := exec.Command(shell, "-NoLogo") + stdin, err := cmd.StdinPipe() + if err != nil { + return fmt.Errorf("error opening stdin pipe: %w", err) + } + + // To slash, so we don't have to escape + bucketsDir := `"` + filepath.ToSlash(scoop.BucketDir()) + `"` + + // FIXME So ... it seems we also need to be able to pass a reference to + // $manifest, which the script CAN manipulate, which we then have to + // reparse. + + go func() { + defer stdin.Close() + for _, line := range lines { + // FIXME Improve implementation + line = strings.ReplaceAll(line, "$bucketsdir", bucketsDir) + fmt.Fprintln(stdin, line) + } + }() + return cmd.Run() +} + +// InstallAll will install the given application into userspace. If an app is +// already installed, it will be updated if applicable. +// +// One key difference to scoop however, is how installing a concrete version +// works. Instead of creating a dirty manifest, we will search for the old +// manifest, install it and hold the app. This will have the same effect for the +// user, but without the fact that the user will never again get update +// notifications. +func (scoop *Scoop) InstallAll(appNames []string, arch ArchitectureKey) []error { + iter := manifestIter() + + var errs []error + for _, inputName := range appNames { + if err := scoop.install(iter, inputName, arch); err != nil { + errs = append(errs, fmt.Errorf("error installing '%s': %w", inputName, err)) + } + } + + return errs +} + +type CacheHit struct { + Downloadable *Downloadable +} + +type FinishedDownload struct { + Downloadable *Downloadable +} + +type StartedDownload struct { + Downloadable *Downloadable +} + +type ChecksumMismatchError struct { + Expected string + Actual string + File string +} + +func (_ *ChecksumMismatchError) Error() string { + return "checksum mismatch" +} + +// Download will download all files for the desired architecture, skipping +// already cached files. The cache lookups happen before downloading and are +// synchronous, directly returning an error instead of using the error channel. +// As soon as download starts (chan, chan, nil) is returned. Both channels are +// closed upon completion (success / failure). +// FIXME Make single result chan with a types: +// (download_start, download_finished, cache_hit) +func (resolvedApp *AppResolved) Download( + cacheDir string, + arch ArchitectureKey, + verifyHashes, overwriteCache bool, +) (chan any, error) { + var download []Downloadable + + // We use a channel for this, as its gonna get more once we finish download + // packages. For downloads, this is not the case, so it is a slice. + results := make(chan any, len(resolvedApp.Downloadables)) + + if overwriteCache { + for _, item := range resolvedApp.Downloadables { + download = append(download, item) + } + } else { + // Parallelise extraction / download. We want to make installation as fast + // as possible. + for _, item := range resolvedApp.Downloadables { + path := filepath.Join( + cacheDir, + CachePath(resolvedApp.Name, resolvedApp.Version, item.URL), + ) + _, err := os.Stat(path) + if err != nil { + if os.IsNotExist(err) { + download = append(download, item) + continue + } + + close(results) + return nil, fmt.Errorf("error checking cached file: %w", err) + } + + if err := validateHash(path, item.Hash); err != nil { + // FIXME We have an error here, but we'll swallow and + // redownload. Should we possibly make a new type? + download = append(download, item) + } else { + results <- &CacheHit{&item} + } + } + } + + cachePath := func(downloadable Downloadable) string { + return filepath.Join(cacheDir, CachePath(resolvedApp.Name, resolvedApp.Version, downloadable.URL)) + } + var requests []*grab.Request + for index, item := range download { + request, err := grab.NewRequest(cachePath(item), item.URL) + if err != nil { + close(results) + return nil, fmt.Errorf("error preparing download: %w", err) + } + + // We attach the item as a context value, since we'll have to make a + // separate mapping otherwise. This is a bit un-nice, but it is stable. + request = request.WithContext(context.WithValue(context.Background(), "item", item)) + request.Label = strconv.FormatInt(int64(index), 10) + requests = append(requests, request) + } + + if len(requests) == 0 { + close(results) + return results, nil + } + + // FIXME Determine batchsize? + client := grab.NewClient() + responses := client.DoBatch(2, requests...) + + // We work on multiple requests at once, but only have one extraction + // routine, as extraction should already make use of many CPU cores. + go func() { + for response := range responses { + if err := response.Err(); err != nil { + results <- fmt.Errorf("error during download: %w", err) + continue + } + + downloadable := response.Request.Context().Value("item").(Downloadable) + results <- &StartedDownload{&downloadable} + + if hashVal := downloadable.Hash; hashVal != "" && verifyHashes { + if err := validateHash(cachePath(downloadable), hashVal); err != nil { + results <- err + continue + } + } + + results <- &FinishedDownload{&downloadable} + } + + close(results) + }() + + return results, nil +} + +func validateHash(path, hashVal string) error { + if hashVal == "" { + return nil + } + + var algo hash.Hash + if strings.HasPrefix(hashVal, "sha1") { + algo = sha1.New() + } else if strings.HasPrefix(hashVal, "sha512") { + algo = sha512.New() + } else if strings.HasPrefix(hashVal, "md5") { + algo = md5.New() + } else { + // sha256 is the default in scoop and has no prefix. This + // will most likely not break, due to the fact scoop goes + // hard on backwards compatibility / not having to migrate + // any of the existing manifests. + algo = sha256.New() + } + + file, err := os.Open(path) + if err != nil { + return fmt.Errorf("error determining checksum: %w", err) + } + + if _, err := io.Copy(algo, file); err != nil { + return fmt.Errorf("error determining checksum: %w", err) + } + + hashVal = strings.ToLower(hashVal) + formattedHash := strings.ToLower(hex.EncodeToString(algo.Sum(nil))) + + if formattedHash != hashVal { + return &ChecksumMismatchError{ + Actual: formattedHash, + Expected: hashVal, + File: path, + } + } + + return nil +} + +func (scoop *Scoop) Install(appName string, arch ArchitectureKey) error { + return scoop.install(manifestIter(), appName, arch) +} + +func (scoop *Scoop) Uninstall(app *InstalledApp, arch ArchitectureKey) error { + resolvedApp := app.ForArch(arch) + + if err := scoop.runScript(resolvedApp.PreUninstall); err != nil { + return fmt.Errorf("error executing pre_uninstall script: %w", err) + } + + if uninstaller := resolvedApp.Uninstaller; uninstaller != nil { + dir := filepath.Join(scoop.AppDir(), app.Name, app.Version) + if err := Installer(*uninstaller).invoke(scoop, dir, arch); err != nil { + return fmt.Errorf("error invoking uninstaller: %w", err) + } + } + + var updatedEnvVars [][2]string + for _, envVar := range resolvedApp.EnvSet { + updatedEnvVars = append(updatedEnvVars, [2]string{envVar.Key, ""}) + } + + if len(resolvedApp.EnvAddPath) > 0 { + pathKey, pathVar, err := windows.GetPersistentEnvValue("User") + if err != nil { + return fmt.Errorf("error retrieving path variable: %w", err) + } + + newPath := windows.ParsePath(pathVar).Remove(resolvedApp.EnvAddPath...) + updatedEnvVars = append(updatedEnvVars, [2]string{pathKey, newPath.String()}) + } + + if err := windows.SetPersistentEnvValues(updatedEnvVars...); err != nil { + return fmt.Errorf("error restoring environment variables: %w", err) + } + + appDir := filepath.Join(scoop.AppDir(), app.Name) + currentDir := filepath.Join(appDir, "current") + + // Make sure installation dir isn't readonly anymore. Scoop does this for + // some reason. + // FIXME The files inside are writable anyway. Should figure out why. + if err := os.Chmod(currentDir, 0o600); err != nil { + return fmt.Errorf("error making current dir deletable: %w", err) + } + + if err := os.RemoveAll(currentDir); err != nil { + return fmt.Errorf("error deleting installation files: %w", err) + } + + if err := scoop.RemoveShims(resolvedApp.Bin...); err != nil { + return fmt.Errorf("error removing shim: %w", err) + } + + // FIXME Do rest of the uninstall here + // 2. Remove shortcuts + + if err := scoop.runScript(resolvedApp.PostUninstall); err != nil { + return fmt.Errorf("error executing post_uninstall script: %w", err) + } + return nil +} + +var ( + ErrAlreadyInstalled = errors.New("app already installed (same version)") + ErrAppNotFound = errors.New("app not found") + ErrAppNotAvailableInVersion = errors.New("app not available in desird version") +) + +func (scoop *Scoop) install(iter *jsoniter.Iterator, appName string, arch ArchitectureKey) error { + fmt.Printf("Installing '%s' ...\n", appName) + + // FIXME Should we check installed first? If it's already installed, we can + // just ignore if it doesn't exist in the bucket anymore. + + app, err := scoop.FindAvailableApp(appName) + if err != nil { + return err + } + + // FIXME Instead try to find it installed / history / workspace. + // Scoop doesnt do this, but we could do it with a "dangerous" flag. + if app == nil { + return ErrAppNotFound + } + + installedApp, err := scoop.FindInstalledApp(appName) + if err != nil { + return fmt.Errorf("error checking for installed version: %w", err) + } + + // FIXME Make force flag. + // FIXME Should this be part of the low level install? + if installedApp != nil && installedApp.Hold { + return fmt.Errorf("app is held: %w", err) + } + + // We might be trying to install a specific version of the given + // application. If this happens, we first look for the manifest in our + // git history. If that fails, we try to auto-generate it. The later is + // what scoop always does. + var manifestFile io.ReadSeeker + _, _, version := ParseAppIdentifier(appName) + if version != "" { + fmt.Printf("Search for manifest version '%s' ...\n", version) + manifestFile, err = app.ManifestForVersion(version) + if err != nil { + return fmt.Errorf("error finding app in version: %w", err) + } + if manifestFile == nil { + return ErrAppNotAvailableInVersion + } + + app = &App{ + Name: app.Name, + Bucket: app.Bucket, + } + if err := app.loadDetailFromManifestWithIter(iter, manifestFile, DetailFieldsAll...); err != nil { + return fmt.Errorf("error loading manifest: %w", err) + } + } else { + manifestFile, err = os.Open(app.ManifestPath()) + if err != nil { + return fmt.Errorf("error opening manifest for copying: %w", err) + } + if err := app.loadDetailFromManifestWithIter(iter, manifestFile, DetailFieldsAll...); err != nil { + return fmt.Errorf("error loading manifest: %w", err) + } + } + + if closer, ok := manifestFile.(io.Closer); ok { + defer closer.Close() + } + + // We reuse the handle. + if _, err := manifestFile.Seek(0, 0); err != nil { + return fmt.Errorf("error resetting manifest file handle: %w", err) + } + + if installedApp != nil { + if err := installedApp.LoadDetailsWithIter(iter, + DetailFieldVersion, + DetailFieldPreUninstall, + DetailFieldPostUninstall, + ); err != nil { + return fmt.Errorf("error determining installed version: %w", err) + } + + // The user should manually run uninstall and install to reinstall. + if installedApp.Version == app.Version { + return ErrAlreadyInstalled + } + + // FIXME Get arch of installed app? Technically we could be on a 64-bit + // system and have the 32-bit version. The same goes for the version + // check. Just because the versions are the same, doesn't mean the arch + // necessarily needs to be the same. + scoop.Uninstall(installedApp, arch) + } + + appDir := filepath.Join(scoop.AppDir(), app.Name) + currentDir := filepath.Join(appDir, "current") + if installedApp != nil { + if err := os.RemoveAll(currentDir); err != nil { + return fmt.Errorf("error removing old currentdir: %w", err) + } + + // FIXME Do rest of the uninstall here + // REmove shims bla bla bla + + scoop.runScript(installedApp.PostUninstall) + } + // FIXME Check if an old version is already installed and we can + // just-relink it. + + resolvedApp := app.ForArch(arch) + + scoop.runScript(resolvedApp.PreInstall) + + versionDir := filepath.Join(appDir, app.Version) + if err := os.MkdirAll(versionDir, os.ModeDir); err != nil { + return fmt.Errorf("error creating isntallation targer dir: %w", err) + } + + cacheDir := scoop.CacheDir() + donwloadResults, err := resolvedApp.Download(cacheDir, arch, true, false) + if err != nil { + return fmt.Errorf("error initialising download: %w", err) + } + + for result := range donwloadResults { + switch result := result.(type) { + case error: + return err + case *CacheHit: + fmt.Printf("Cache hit for '%s'", filepath.Base(result.Downloadable.URL)) + if err := scoop.extract(app, resolvedApp, cacheDir, versionDir, *result.Downloadable, arch); err != nil { + return fmt.Errorf("error extracting file '%s': %w", filepath.Base(result.Downloadable.URL), err) + } + case *FinishedDownload: + fmt.Printf("Downloaded '%s'\n", filepath.Base(result.Downloadable.URL)) + if err := scoop.extract(app, resolvedApp, cacheDir, versionDir, *result.Downloadable, arch); err != nil { + return fmt.Errorf("error extracting file '%s': %w", filepath.Base(result.Downloadable.URL), err) + } + } + } + + if installer := resolvedApp.Installer; installer != nil { + dir := filepath.Join(scoop.AppDir(), app.Name, app.Version) + if err := installer.invoke(scoop, dir, arch); err != nil { + return fmt.Errorf("error invoking installer: %w", err) + } + } + + // FIXME Make copy util? + // FIXME Read perms? + newManifestFile, err := os.OpenFile( + filepath.Join(versionDir, "manifest.json"), os.O_CREATE|os.O_TRUNC, 0o600) + if err != nil { + return fmt.Errorf("error creating new manifest: %w", err) + } + if _, err := io.Copy(newManifestFile, manifestFile); err != nil { + return fmt.Errorf("error copying manfiest: %w", err) + } + + fmt.Println("Linking to newly installed version.") + if err := windows.CreateJunctions([2]string{versionDir, currentDir}); err != nil { + return fmt.Errorf("error linking from new current dir: %w", err) + } + + // Shims are copies of a certain binary that uses a ".shim" file next to + // it to realise some type of symlink. + for _, bin := range resolvedApp.Bin { + fmt.Printf("Creating shim for '%s'\n", bin.Name) + if err := scoop.CreateShim(filepath.Join(currentDir, bin.Name), bin); err != nil { + return fmt.Errorf("error creating shim: %w", err) + } + } + + var envVars [][2]string + if len(resolvedApp.EnvAddPath) > 0 { + pathKey, oldPath, err := windows.GetPersistentEnvValue("Path") + if err != nil { + return fmt.Errorf("error attempt to add variables to path: %w", err) + } + parsedPath := windows.ParsePath(oldPath).Prepend(resolvedApp.EnvAddPath...) + envVars = append(envVars, [2]string{pathKey, parsedPath.String()}) + } + + for _, pathEntry := range resolvedApp.EnvSet { + value := substituteVariables(pathEntry.Value, map[string]string{ + "dir": currentDir, + "persist_dir": filepath.Join(scoop.PersistDir(), app.Name), + }) + envVars = append(envVars, [2]string{pathEntry.Key, value}) + } + + if err := windows.SetPersistentEnvValues(envVars...); err != nil { + return fmt.Errorf("error setting env values: %w", err) + } + + // FIXME Adjust arch value if we install anything else than is desired. + if err := os.WriteFile(filepath.Join(versionDir, "install.json"), []byte(fmt.Sprintf( + `{ + "bucket": "%s", + "architecture": "%s", + "hold": %v +}`, app.Bucket.Name(), arch, version != "")), 0o600); err != nil { + return fmt.Errorf("error writing installation information: %w", err) + } + + if err := scoop.runScript(resolvedApp.PostInstall); err != nil { + return fmt.Errorf("error running post install script: %w", err) + } + + return nil +} + +func substituteVariables(value string, variables map[string]string) string { + // It seems like scoop does it this way as well. Instead of somehow checking + // whether there's a variable such as $directory, we simply replace $dir, + // not paying attention to potential damage done. + // FIXME However, this is error prone and should change in the future. + for key, val := range variables { + value = strings.ReplaceAll(value, key, val) + } + + // FIXME Additionally, we need to substitute any $env:VARIABLE. The bullet + // proof way to do this, would be to simply invoke powershell, albeit a bit + // slow. This should happen before the in-code substitution. + + // This needs more investigation though, should probably read the docs on + // powershell env var substitution and see how easy it would be. + + return value +} + +// extract will extract the given item. It doesn't matter which type it has, as +// this function will call the correct function. For example, a `.msi` will +// cause invocation of `lessmesi`. Note however, that this function isn't +// thread-safe, as it might install additional tooling required for extraction. +func (scoop *Scoop) extract( + app *App, + resolvedApp *AppResolved, + cacheDir string, + appDir string, + item Downloadable, + arch ArchitectureKey, +) error { + baseName := filepath.Base(item.URL) + fmt.Printf("Extracting '%s' ...\n", baseName) + + fileToExtract := filepath.Join(cacheDir, CachePath(app.Name, app.Version, item.URL)) + targetPath := filepath.Join(appDir, item.ExtractTo) + + // Depending on metadata / filename, we decide how to extract the + // files that are to be installed. Note we don't care whether the + // dependency is installed via scoop, we just want it to be there. + + // We won't even bother testing the extension here, as it could + // technically be an installed not ending on `.exe`. While this is + // not true for the other formats, it is TECHNCIALLY possible here. + if resolvedApp.InnoSetup { + // If this flag is set, the installer.script might be set, but the + // installer.file never is, meaning extraction is always the correct + // thing to do. + + innounpPath, err := exec.LookPath("innounp") + if err == nil && innounpPath != "" { + goto INVOKE_INNOUNP + } + if err != nil { - return fmt.Errorf("error installing app '%s': %w", inputName, err) + return fmt.Errorf("error looking up innounp: %w", err) + } + + if err := scoop.Install("innounp", arch); err != nil { + return fmt.Errorf("error installing dependency innounp: %w", err) } - // FIXME Instead try to find it installed / history / workspace. - // Scoop doesnt do this, but we could do it with a "dangerous" flag. - if app == nil { - return fmt.Errorf("app '%s' not found", inputName) + INVOKE_INNOUNP: + args := []string{ + // Extract + "-x", + // Confirm questions + "-y", + // Destination + "-d" + targetPath, + fileToExtract, } - // We might be trying to install a specific version of the given - // application. If this happens, we first look for the manifest in our - // git history. If that fails, we try to auto-generate it. The later is - // what scoop always does. - _, _, version := ParseAppIdentifier(inputName) - if version != "" { + if strings.HasPrefix(item.ExtractDir, "{") { + args = append(args, "-c"+item.ExtractDir) + } else if item.ExtractDir != "" { + args = append(args, "-c{app}\\"+item.ExtractDir) + } else { + args = append(args, "-c{app}") } + + cmd := exec.Command("innounp", args...) + if err := cmd.Run(); err != nil { + return fmt.Errorf("error invoking innounp: %w", err) + } + + return nil } + ext := strings.ToLower(filepath.Ext(item.URL)) + // 7zip supports A TON of file formats, so we try to use it where we + // can. It's fast and known to work well. + if supportedBy7Zip(ext) { + sevenZipPath, err := exec.LookPath("7z") + // Path can be non-empty and still return an error. Read + // LookPath documentation. + if err == nil && sevenZipPath != "" { + goto INVOKE_7Z + } + + // Fallback for cases where we don't have 7zip installed, but still + // want to unpack a zip. Without this, we'd print an error instead. + if ext == ".zip" { + goto STD_ZIP + } + + if err != nil { + return fmt.Errorf("error doing path lookup: %w", err) + } + + if err := scoop.Install("7zip", arch); err != nil { + return fmt.Errorf("error installing dependency 7zip: %w", err) + } + + INVOKE_7Z: + args := []string{ + // Extract from file + "x", + fileToExtract, + // Target path + "-o" + targetPath, + // Overwrite all files + "-aoa", + // Confirm + "-y", + } + // FIXME: $IsTar = ((strip_ext $Path) -match '\.tar$') -or ($Path -match '\.t[abgpx]z2?$') + if ext != ".tar" && item.ExtractDir != "" { + args = append(args, "-ir!"+filepath.Join(item.ExtractDir, "*")) + } + cmd := exec.Command( + "7z", + args..., + ) + if err := cmd.Run(); err != nil { + return fmt.Errorf("error invoking 7z: %w", err) + } + } + + // TODO: dark, msi, inno, installer, zst + + switch ext { + case ".msi": + lessmsiPath, err := scoop.ensureExecutable("lessmsi", "lessmsi", arch) + if err != nil { + return fmt.Errorf("error installing lessmsi: %w", err) + } + fmt.Println(lessmsiPath) + + return nil + } + +STD_ZIP: + if ext == ".zip" { + zipReader, err := zip.OpenReader(fileToExtract) + if err != nil { + return fmt.Errorf("error opening zip reader: %w", err) + } + + for _, f := range zipReader.File { + // We create these anyway later. + if f.FileInfo().IsDir() { + continue + } + + // FIXME Prevent accidental mismatches + extractDir := filepath.ToSlash(item.ExtractDir) + fName := filepath.ToSlash(f.Name) + if extractDir != "" && !strings.HasPrefix(fName, extractDir) { + continue + } + + // Strip extract dir, as these aren't meant to be preserved, + // unless specified via extractTo + fName = strings.TrimLeft(strings.TrimPrefix(fName, extractDir), "/") + + filePath := filepath.Join(appDir, item.ExtractTo, fName) + if err := os.MkdirAll(filepath.Dir(filePath), os.ModePerm); err != nil { + return fmt.Errorf("error creating dir: %w", err) + } + + dstFile, err := os.OpenFile(filePath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, f.Mode()) + if err != nil { + return fmt.Errorf("error creating target file for zip entry: %w", err) + } + + fileInArchive, err := f.Open() + if err != nil { + return fmt.Errorf("error opening zip file entry: %w", err) + } + + if _, err := io.Copy(dstFile, fileInArchive); err != nil { + return fmt.Errorf("error copying zip file entry: %w", err) + } + + dstFile.Close() + fileInArchive.Close() + } + } else { + targetFile, err := os.OpenFile( + filepath.Join(appDir, item.ExtractTo, baseName), + os.O_CREATE|os.O_WRONLY|os.O_TRUNC, + 0o600, + ) + if err != nil { + return fmt.Errorf("error opening handle target file: %w", err) + } + defer targetFile.Close() + + sourceFile, err := os.Open(fileToExtract) + if err != nil { + return fmt.Errorf("error opening cache file: %w", err) + } + defer sourceFile.Close() + + if _, err := io.Copy(targetFile, sourceFile); err != nil { + return fmt.Errorf("error copying file: %w", err) + } + } + + // Mark RO afterwards? return nil } +// ensureExecutable will look for a given executable on the path. If not +// found, it will attempt installing the dependency using the given app +// information. +func (scoop *Scoop) ensureExecutable(executable, appName string, arch ArchitectureKey) (string, error) { + executablePath, err := exec.LookPath(executable) + if err != nil { + if !errors.Is(err, exec.ErrDot) && !errors.Is(err, exec.ErrNotFound) { + return "", fmt.Errorf("error locating '%s': %w", executable, err) + } + + // We'll treat a relative path binary as non-existent for now and + // install the dependency. + executablePath = "" + } + + if executablePath == "" { + if err := scoop.Install(appName, arch); err != nil { + return "", fmt.Errorf("error installing required dependency '%s': %w", appName, err) + } + + executablePath, err = exec.LookPath(executable) + if err != nil { + return "", fmt.Errorf("error locating '%s': %w", executable, err) + } + } + + // Might be empty if the second lookup failed. HOWEVER, it shouldn't as we + // simply add to the shims folder, which should already be on the path. + return executablePath, err +} + +var sevenZipFileFormatRegex = regexp.MustCompile(`\.((gz)|(tar)|(t[abgpx]z2?)|(lzma)|(bz2?)|(7z)|(001)|(rar)|(iso)|(xz)|(lzh)|(nupkg))(\.[^\d.]+)?$`) + +func supportedBy7Zip(extension string) bool { + return sevenZipFileFormatRegex.MatchString(extension) +} + +// AppResolved is a version of app forming the data into a way that it's ready +// for installation, deinstallation or update. +type AppResolved struct { + *App + + // TODO checkver, hash, extract_dir; + // TODO Merge url, hash and extract_dir? Like we did with bin, to give + // semantic meaning to it. + + Bin []Bin `json:"bin"` + Shortcuts []Bin `json:"shortcuts"` + + Downloadables []Downloadable `json:"downloadables"` + + // Installer deprecates msi; InnoSetup bool should be same for each + // architecture. The docs don't mention it. + Installer *Installer `json:"installer"` + PreInstall []string `json:"pre_install"` + PostInstall []string `json:"post_install"` +} + +// ForArch will create a merged version that includes all the relevant fields at +// root level. Access to architecture shouldn't be required anymore, it should +// be ready to use for installtion, update or uninstall. +func (a *App) ForArch(arch ArchitectureKey) *AppResolved { + resolved := &AppResolved{ + App: a, + } + + resolved.Bin = a.Bin + resolved.Shortcuts = a.Shortcuts + resolved.Downloadables = a.Downloadables + resolved.PreInstall = a.PreInstall + resolved.PostInstall = a.PostInstall + resolved.Installer = a.Installer + + if a.Architecture == nil { + return resolved + } + + archValue := a.Architecture[arch] + if archValue == nil && arch == ArchitectureKey64Bit { + // Fallbackt to 32bit. If we are on arm, there's no use to fallback + // though, since only arm64 is supported by scoop either way. + archValue = a.Architecture[ArchitectureKey32Bit] + } + if archValue != nil { + // nil-checking might be fragile, so this is safer. + if len(archValue.Bin) > len(resolved.Bin) { + resolved.Bin = archValue.Bin + } + if len(archValue.Shortcuts) > len(resolved.Shortcuts) { + resolved.Shortcuts = archValue.Shortcuts + } + if len(archValue.Downloadables) > len(resolved.Downloadables) { + // If we need to manipulate these, we do a copy, to prevent changing the + // opriginal app. + if len(a.ExtractTo) > 0 { + resolved.Downloadables = append([]Downloadable{}, archValue.Downloadables...) + } else { + resolved.Downloadables = archValue.Downloadables + } + } + if len(archValue.PreInstall) > len(resolved.PreInstall) { + resolved.PreInstall = archValue.PreInstall + } + if len(archValue.PostInstall) > len(resolved.PostInstall) { + resolved.PostInstall = archValue.PostInstall + } + } + + // architecture does not support extract_to, so we merge it with the root + // level value for ease of use. + switch len(a.ExtractTo) { + case 0: + // Do nothing, path inferred to app root dir (current). + case 1: + // Same path everywhere + for i := 0; i < len(resolved.Downloadables); i++ { + resolved.Downloadables[i].ExtractTo = a.ExtractTo[0] + } + default: + // Path per URL, but to be defensive, we'll infer if missing ones, by + // leaving it empty (current root dir). + for i := 0; i < len(resolved.Downloadables) && i < len(a.ExtractTo); i++ { + resolved.Downloadables[i].ExtractTo = a.ExtractTo[i] + } + } + + // If we have neither an installer file, nor a script, we reference the last + // items downloaded, as per scoop documentation. + // FIXME Find out if this is really necessary, this is jank. + if a.Installer != nil && a.Installer.File == "" && + len(a.Installer.Script) == 0 && len(a.Downloadables) > 0 { + lastURL := resolved.Downloadables[len(a.Downloadables)-1].URL + a.Installer.File = filepath.Base(lastURL) + } + + return resolved +} + var ErrBucketNoGitDir = errors.New(".git dir at path not found") func (a *App) AvailableVersions() ([]string, error) { @@ -847,7 +1545,7 @@ func readVersion(iter *jsoniter.Iterator, data []byte) string { // desired version is found. Note that we compare the versions and stop // searching if a lower version is encountered. This function is expected to // be very slow, be warned! -func (a *App) ManifestForVersion(targetVersion string) (io.ReadCloser, error) { +func (a *App) ManifestForVersion(targetVersion string) (io.ReadSeeker, error) { repoPath, relManifestPath := git.GitPaths(a.ManifestPath()) if repoPath == "" || relManifestPath == "" { return nil, ErrBucketNoGitDir @@ -874,7 +1572,7 @@ func (a *App) ManifestForVersion(targetVersion string) (io.ReadCloser, error) { version := readVersion(iter, result.Data) comparison := versioncmp.Compare(version, targetVersion, cmpRules) if comparison == "" { - return io.NopCloser(bytes.NewReader(result.Data)), nil + return bytes.NewReader(result.Data), nil } // The version we are looking for is greater than the one from history, @@ -896,7 +1594,7 @@ func (scoop *Scoop) LookupCache(app, version string) ([]string, error) { expectedPrefix += "#" + cachePathRegex.ReplaceAllString(version, "_") } - return filepath.Glob(filepath.Join(scoop.GetCacheDir(), expectedPrefix+"*")) + return filepath.Glob(filepath.Join(scoop.CacheDir(), expectedPrefix+"*")) } var cachePathRegex = regexp.MustCompile(`[^\w\.\-]+`) @@ -911,7 +1609,7 @@ func CachePath(app, version, url string) string { return strings.Join(parts, "#") } -func (scoop *Scoop) GetCacheDir() string { +func (scoop *Scoop) CacheDir() string { return filepath.Join(scoop.scoopRoot, "cache") } @@ -919,7 +1617,7 @@ type Scoop struct { scoopRoot string } -func (scoop *Scoop) GetAppsDir() string { +func (scoop *Scoop) AppDir() string { return filepath.Join(scoop.scoopRoot, "apps") } diff --git a/pkg/scoop/scoop_test.go b/pkg/scoop/scoop_test.go index 7cb11fd..db454d6 100644 --- a/pkg/scoop/scoop_test.go +++ b/pkg/scoop/scoop_test.go @@ -11,7 +11,7 @@ func app(t *testing.T, name string) *scoop.App { defaultScoop, err := scoop.NewScoop() require.NoError(t, err) - app, err := defaultScoop.GetAvailableApp(name) + app, err := defaultScoop.FindAvailableApp(name) require.NoError(t, err) return app @@ -21,7 +21,7 @@ func Test_ManifestForVersion(t *testing.T) { defaultScoop, err := scoop.NewScoop() require.NoError(t, err) - app, err := defaultScoop.GetAvailableApp("main/go") + app, err := defaultScoop.FindAvailableApp("main/go") require.NoError(t, err) t.Run("found", func(t *testing.T) { @@ -78,6 +78,26 @@ func Test_ParseBin(t *testing.T) { Args: []string{`-c "$dir\config\config.yml"`}, }) }) + t.Run("nested array that contains arrays and strings", func(t *testing.T) { + app := app(t, "main/python") + + err := app.LoadDetails(scoop.DetailFieldBin) + require.NoError(t, err) + + // Order doesnt matter + require.Len(t, app.Bin, 3) + require.Contains(t, app.Bin, scoop.Bin{ + Name: "python.exe", + Alias: "python3", + }) + require.Contains(t, app.Bin, scoop.Bin{ + Name: "Lib\\idlelib\\idle.bat", + }) + require.Contains(t, app.Bin, scoop.Bin{ + Name: "Lib\\idlelib\\idle.bat", + Alias: "idle3", + }) + }) } func Test_ParseArchitecture_Items(t *testing.T) { @@ -95,20 +115,20 @@ func Test_ParseArchitecture_Items(t *testing.T) { arm64 := arch[scoop.ArchitectureKeyARM64] require.NotNil(t, arm64) - require.Len(t, x386.Items, 1) - require.Len(t, x686.Items, 1) - require.Len(t, arm64.Items, 1) + require.Len(t, x386.Downloadables, 1) + require.Len(t, x686.Downloadables, 1) + require.Len(t, arm64.Downloadables, 1) - require.Contains(t, x386.Items[0].URL, "386") - require.NotEmpty(t, x386.Items[0].Hash) - require.Empty(t, x386.Items[0].ExtractDir) + require.Contains(t, x386.Downloadables[0].URL, "386") + require.NotEmpty(t, x386.Downloadables[0].Hash) + require.Empty(t, x386.Downloadables[0].ExtractDir) - require.Contains(t, x686.Items[0].URL, "amd64") - require.NotEmpty(t, x686.Items[0].Hash) - require.Empty(t, x686.Items[0].ExtractDir) + require.Contains(t, x686.Downloadables[0].URL, "amd64") + require.NotEmpty(t, x686.Downloadables[0].Hash) + require.Empty(t, x686.Downloadables[0].ExtractDir) - require.Contains(t, arm64.Items[0].URL, "arm64") - require.NotEmpty(t, arm64.Items[0].URL) - require.NotEmpty(t, arm64.Items[0].Hash) - require.Empty(t, arm64.Items[0].ExtractDir) + require.Contains(t, arm64.Downloadables[0].URL, "arm64") + require.NotEmpty(t, arm64.Downloadables[0].URL) + require.NotEmpty(t, arm64.Downloadables[0].Hash) + require.Empty(t, arm64.Downloadables[0].ExtractDir) } diff --git a/pkg/scoop/shim.exe b/pkg/scoop/shim.exe new file mode 100644 index 0000000..3ab79dd Binary files /dev/null and b/pkg/scoop/shim.exe differ diff --git a/pkg/scoop/shim.go b/pkg/scoop/shim.go new file mode 100644 index 0000000..cdfb8a4 --- /dev/null +++ b/pkg/scoop/shim.go @@ -0,0 +1,152 @@ +package scoop + +import ( + "bytes" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" + + _ "embed" +) + +//go:embed shim_cmd_to_cmd.template +var cmdToCmdTemplate string + +//go:embed shim_cmd_to_bash.template +var cmdToBashTemplate string + +//go:embed shim.exe +var shimExecutable []byte + +// FIXME Should this be a public helper function on Bin? If so, we should +// probably split bin and shortcut. At this point, they don't seem to be to +// compatible anymore. +func shimName(bin Bin) string { + shimName := bin.Alias + if shimName == "" { + shimName = filepath.Base(bin.Name) + shimName = strings.TrimSuffix(shimName, filepath.Ext(shimName)) + } + return shimName +} + +func (scoop *Scoop) RemoveShims(bins ...Bin) error { + return filepath.WalkDir(scoop.ShimDir(), func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + + if d.IsDir() { + return nil + } + + for _, bin := range bins { + // This will catch all file types, including the shims. + shimName := shimName(bin) + binWithoutExt := strings.TrimSuffix(shimName, filepath.Ext(shimName)) + nameWithoutExt := strings.TrimSuffix(d.Name(), filepath.Ext(d.Name())) + if !strings.EqualFold(nameWithoutExt, binWithoutExt) { + continue + } + + if err := os.Remove(path); err != nil { + return fmt.Errorf("error deleting shim '%s': %w", path, err) + } + } + + return nil + }) +} + +func (scoop *Scoop) CreateShim(path string, bin Bin) error { + /* + We got the following possible constructs: + + 0. + bin: [ + path/to/file + ] + 1. + bin: [ + [ + path/to/file + shim.type + ] + ] + 2. + bin: [ + [ + path/to/file.exe + shim + ] + ] + + In case 0. we simply create whatever extension the file had as a + shim, falling back to .cmd. + + In case 1. we create a shim given the desired extension, no matter + what extension the actual file has. The same goes for case 2. where we + haven't passed an explicit shim extension even though we know it's an + executable. + */ + + shimName := shimName(bin) + + switch filepath.Ext(bin.Name) { + case ".exe", ".com": + // FIXME Do we need to escape anything here? + argsJoined := strings.Join(bin.Args, " ") + + // The .shim and .exe files needs to be writable, as scoop fails to + // uninstall otherwise. + var shimConfig bytes.Buffer + shimConfig.WriteString(`path = "`) + shimConfig.WriteString(path) + shimConfig.WriteString("\"\n") + if argsJoined != "" { + shimConfig.WriteString(`args = `) + shimConfig.WriteString(argsJoined) + shimConfig.WriteRune('\n') + } + if err := os.WriteFile(filepath.Join(scoop.ShimDir(), shimName+".shim"), + shimConfig.Bytes(), 0o600); err != nil { + return fmt.Errorf("error writing shim file: %w", err) + } + + targetPath := filepath.Join(scoop.ShimDir(), shimName+".exe") + err := os.WriteFile(targetPath, shimExecutable, 0o700) + if err != nil { + return fmt.Errorf("error creating shim executable: %w", err) + } + case ".cmd", ".bat": + // FIXME Do we need to escape anything here? + argsJoined := strings.Join(bin.Args, " ") + + if err := os.WriteFile( + filepath.Join(scoop.ShimDir(), shimName+".cmd"), + []byte(fmt.Sprintf(cmdToCmdTemplate, path, path, argsJoined)), + 0o700, + ); err != nil { + return fmt.Errorf("error creating cmdShim: %w", err) + } + if err := os.WriteFile( + filepath.Join(scoop.ShimDir(), shimName), + []byte(fmt.Sprintf(cmdToBashTemplate, path, path, argsJoined)), + 0o700, + ); err != nil { + return fmt.Errorf("error creating cmdShim: %w", err) + } + case ".ps1": + case ".jar": + case ".py": + default: + } + + return nil +} + +func (scoop *Scoop) ShimDir() string { + return filepath.Join(scoop.scoopRoot, "shims") +} diff --git a/pkg/scoop/shim_bash.template b/pkg/scoop/shim_bash.template new file mode 100644 index 0000000..e69de29 diff --git a/pkg/scoop/shim_cmd_to_bash.template b/pkg/scoop/shim_cmd_to_bash.template new file mode 100644 index 0000000..66aeacc --- /dev/null +++ b/pkg/scoop/shim_cmd_to_bash.template @@ -0,0 +1,4 @@ +#!/bin/sh +# %s +echo "bashin" +MSYS2_ARG_CONV_EXCL=/C cmd.exe /C "%s" %s "$@" diff --git a/pkg/scoop/shim_cmd_to_cmd.template b/pkg/scoop/shim_cmd_to_cmd.template new file mode 100644 index 0000000..e80069c --- /dev/null +++ b/pkg/scoop/shim_cmd_to_cmd.template @@ -0,0 +1,3 @@ +@rem %s +@"%s" %* + diff --git a/pkg/scoop/shim_ps1_to_ps1.template b/pkg/scoop/shim_ps1_to_ps1.template new file mode 100644 index 0000000..d01c611 --- /dev/null +++ b/pkg/scoop/shim_ps1_to_ps1.template @@ -0,0 +1,5 @@ +# %s +$path = "%s" +if ($MyInvocation.ExpectingInput) { $input | & $path $arg @args } else { & $path $arg @args } +exit $LASTEXITCODE + diff --git a/pkg/scoop/shim_sh.template b/pkg/scoop/shim_sh.template new file mode 100644 index 0000000..e69de29