diff --git a/crates/moonwell-deploy/README.md b/crates/moonwell-deploy/README.md index bfad45584..a49fc9ae6 100644 --- a/crates/moonwell-deploy/README.md +++ b/crates/moonwell-deploy/README.md @@ -44,8 +44,9 @@ Optional `-PatchSource` points to your own source tree containing `Data/patch-M/`, `Data/patch-N/`, etc. The Rust builder produces `Data/patch-M.MPQ` and mounts them through the client's vanilla patch chain. A supplied patch source takes precedence over the same named patch in the input install. -Locale subdirectories such as `Data/ruRU/patch-ruRU-5` are rejected because -MoonWell currently has no locale selection or mounting support. +The mounted archives under `Data/ruRU` are included as shared language files. +The game chooses the installed language through `WOW_LOCALE=auto`. Locale patch +source directories are not supported by the patch builder. The Windows script includes `vcruntime140.dll` from Visual Studio's x64 redistributable directory beside the game executable. `-VcRuntimePath` can @@ -112,14 +113,60 @@ no S3 credentials. Inventories without `objectKey` retain the legacy `/` resolution. The old 3.3.5 `manifest.json` and `World of Warcraft/` keys are reserved and cannot be publication targets. -Native macOS binaries must be built on a Mac and packaged with -[`package_macos.py`](../moonwell/package_macos.py). `stage --binary MoonWell.app` -can stage such a bundle, but publish it with separate `--manifest-key` and -`--prefix` settings for a launcher distribution targeting macOS. The current -API has one active inventory per configured distribution, not automatic OS -selection. This Windows pipeline does not build or sign macOS applications. -The current Flutter file downloader also does not restore Unix executable bits; -install the `.app` as a bundle with its permissions preserved when testing macOS. +## Shared data and Apple Silicon publication + +Publish common data independently from binary releases: + +```sh +cargo run -p moonwell-deploy -- stage-data --data /path/to/Data \ + --output dist/common --auth-host login.example:3724 +cargo run -p moonwell-deploy -- publish-data --dir dist/common --env-file /path/to/.env --dry-run +# Repeat without --dry-run to switch manifest-1.12.1-data.json. +``` + +This uses the same archive and agreement validation as full staging. Shared data +contains only `Data/` and `realmlist.wtf`. To migrate without re-uploading data, +initial Mac publication may use `--shared-manifest-key manifest-1.12.1.json`: +only common files are selected from that existing Windows inventory. + +On an Apple Silicon Mac with the Rust target installed and Python 3.11+: + +```sh +rustup target add aarch64-apple-darwin +python3 scripts/deploy-moonwell-macos.py --icon /path/to/MoonWell.icns +python3 scripts/deploy-moonwell-macos.py --icon /path/to/MoonWell.icns \ + --env-file /path/to/.env --dry-run +# Use --publish for publication; --binary can reuse an existing arm64 player build. +``` + +The script builds without development features, creates and ad hoc signs the app, +rejects unbundled non-system libraries, and writes `MoonWell.app.tar.gz` and +`bundle.json`. The archive is restricted USTAR with regular files only. Its +inventory records installed content hashes and executable flags. No paid signing +identity or notarization is used; users may need macOS security approval. + +`publish-macos` composes shared references with the bundle and switches +`manifest-1.12.1-macos-arm64.json`. Binary-only updates require no local data copy +and no shared archive uploads. Referenced objects are checked for existence and +size. All objects remain inside the common client storage prefix. + +Windows binary-only publication can use the existing `publish` command with +`--shared-manifest-key manifest-1.12.1-data.json`. Its input directory must contain +only `MoonWell.exe` and required runtime DLLs. The full Windows staging script +continues working unchanged. + +Publishing new common data does not silently activate it for every platform. +Recompose each platform inventory with its selected binaries and the new shared +inventory when ready. Retained objects permit independent rollback. + +New inventories include `platform` and `release`. Windows keeps the installed-file +`buildHash` as its release ID. Mac release IDs are SHA-256 of +`:`; buildHash covers shared files plus installed bundle +files using the existing canonical file hash. The immutable inventory is written +at `/releases//manifest.json` before the active pointer is updated. +The Laravel backend and launcher use `platform` and `release` on downloads so an +active publication cannot change an in-progress sync. Deploy compatible backend +and launcher versions before activating Mac inventories. ## Checks diff --git a/crates/moonwell-deploy/src/main.rs b/crates/moonwell-deploy/src/main.rs index 94d8760a8..535ebdea2 100644 --- a/crates/moonwell-deploy/src/main.rs +++ b/crates/moonwell-deploy/src/main.rs @@ -5,7 +5,7 @@ mod vanilla_mpq; use std::path::PathBuf; -use anyhow::{Context, Result}; +use anyhow::{ensure, Context, Result}; use clap::{Parser, Subcommand}; #[derive(Parser)] @@ -37,6 +37,45 @@ enum Command { #[arg(long)] auth_host: String, }, + /// Stage common Data and realmlist without a platform executable. + StageData { + #[arg(long)] + data: PathBuf, + #[arg(long)] + output: PathBuf, + #[arg(long)] + source: Option, + #[arg(long)] + auth_host: String, + }, + /// Publish common data independently of game binaries. + PublishData { + #[arg(long)] + dir: PathBuf, + #[arg(long)] + env_file: PathBuf, + #[arg(long, default_value = "World of Warcraft 1.12.1")] + prefix: String, + #[arg(long, default_value = "manifest-1.12.1-data.json")] + manifest_key: String, + #[arg(long)] + dry_run: bool, + }, + /// Publish a packaged Mac archive with existing common data references. + PublishMacos { + #[arg(long)] + dir: PathBuf, + #[arg(long)] + env_file: PathBuf, + #[arg(long, default_value = "World of Warcraft 1.12.1")] + prefix: String, + #[arg(long, default_value = "manifest-1.12.1-macos-arm64.json")] + manifest_key: String, + #[arg(long, default_value = "manifest-1.12.1-data.json")] + shared_manifest_key: String, + #[arg(long)] + dry_run: bool, + }, /// Generate the launcher inventory of an existing distribution directory. Manifest { #[arg(long)] @@ -66,6 +105,9 @@ enum Command { prefix: String, #[arg(long, default_value = "manifest-1.12.1.json")] manifest_key: String, + /// Compose Windows binaries with a published common data inventory. + #[arg(long)] + shared_manifest_key: Option, /// Read and report the upload plan without writing any S3 objects. #[arg(long)] dry_run: bool, @@ -82,6 +124,53 @@ fn run() -> Result<()> { source, auth_host, } => package::stage(&binary, &data, &output, source.as_deref(), &auth_host), + Command::StageData { + data, + output, + source, + auth_host, + } => package::stage_data(&data, &output, source.as_deref(), &auth_host), + Command::PublishData { + dir, + env_file, + prefix, + manifest_key, + dry_run, + } => { + package::validate_shared_distribution(&dir)?; + let mut m = manifest::scan(&dir)?; + ensure!( + m.files + .iter() + .all(|f| f.path.starts_with("Data/") || f.path == "realmlist.wtf"), + "data package contains platform files" + ); + m.platform = Some("shared".into()); + let store = storage::S3Store::from_env(&env_file)?; + storage::publish(&store, &dir, m, &prefix, &manifest_key, dry_run) + } + Command::PublishMacos { + dir, + env_file, + prefix, + manifest_key, + shared_manifest_key, + dry_run, + } => { + let store = storage::S3Store::from_env(&env_file)?; + let shared = load_shared(&store, &shared_manifest_key, &prefix)?; + let bundle: manifest::ClientBundle = + serde_json::from_slice(&std::fs::read(dir.join("bundle.json"))?)?; + let mut m = manifest::Manifest { + build_hash: String::new(), + files: shared, + platform: Some("macos-arm64".into()), + release: None, + bundle: Some(bundle), + }; + m.build_hash = manifest::build_hash(&m.installed_files()); + storage::publish(&store, &dir, m, &prefix, &manifest_key, dry_run) + } Command::Manifest { dir, output } => { package::validate_distribution(&dir)?; let m = manifest::scan(&dir)?; @@ -122,10 +211,26 @@ fn run() -> Result<()> { prefix, manifest_key, dry_run, + shared_manifest_key, } => { - package::validate_distribution(&dir)?; - let m = manifest::scan(&dir)?; let store = storage::S3Store::from_env(&env_file)?; + let mut m = manifest::scan(&dir)?; + if let Some(key) = shared_manifest_key { + ensure!( + m.files + .iter() + .all(|f| f.path == "MoonWell.exe" || f.path.ends_with(".dll")), + "Windows binary package contains unexpected files" + ); + let mut magic = [0; 2]; + use std::io::Read; + std::fs::File::open(dir.join("MoonWell.exe"))?.read_exact(&mut magic)?; + ensure!(magic == *b"MZ", "Windows player executable is missing"); + m.files.extend(load_shared(&store, &key, &prefix)?); + m.build_hash = manifest::build_hash(&m.files); + } else { + package::validate_distribution(&dir)?; + } storage::publish(&store, &dir, m, &prefix, &manifest_key, dry_run) } } @@ -137,3 +242,16 @@ fn main() { std::process::exit(1); } } + +fn load_shared( + store: &impl storage::Store, + key: &str, + prefix: &str, +) -> Result> { + ensure!(manifest::valid_path(key), "invalid shared manifest key"); + let bytes = store + .get(key)? + .context("shared inventory not found; publish data first")?; + let shared: manifest::Manifest = serde_json::from_slice(&bytes)?; + manifest::shared_files(&shared, prefix) +} diff --git a/crates/moonwell-deploy/src/manifest.rs b/crates/moonwell-deploy/src/manifest.rs index 771d7dfd1..aaee95d1f 100644 --- a/crates/moonwell-deploy/src/manifest.rs +++ b/crates/moonwell-deploy/src/manifest.rs @@ -13,6 +13,40 @@ use walkdir::WalkDir; pub struct Manifest { pub build_hash: String, pub files: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub platform: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub release: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub bundle: Option, +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct ClientBundle { + #[serde(flatten)] + pub archive: ClientFile, + pub files: Vec, +} + +impl Manifest { + pub fn installed_files(&self) -> Vec { + let mut files = self.files.clone(); + if let Some(bundle) = &self.bundle { + files.extend(bundle.files.clone()); + } + files + } + + pub fn release_id(&self) -> String { + match &self.bundle { + Some(bundle) => format!( + "{:x}", + Sha256::digest(format!("{}:{}", self.build_hash, bundle.archive.sha256)) + ), + None => self.build_hash.clone(), + } + } } #[derive(Clone, Debug, Deserialize, Serialize)] @@ -23,6 +57,8 @@ pub struct ClientFile { pub sha256: String, #[serde(default, skip_serializing_if = "Option::is_none")] pub object_key: Option, + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub executable: bool, } pub fn valid_path(path: &str) -> bool { @@ -116,6 +152,7 @@ pub fn scan(root: &Path) -> Result { size, sha256, object_key: None, + executable: false, }); } files.sort_by(|a, b| a.path.cmp(&b.path)); @@ -123,13 +160,16 @@ pub fn scan(root: &Path) -> Result { Ok(Manifest { build_hash: build_hash(&files), files, + platform: None, + release: None, + bundle: None, }) } pub fn validate(manifest: &Manifest) -> Result<()> { ensure!(!manifest.files.is_empty(), "manifest is empty"); let mut paths = HashSet::new(); - for f in &manifest.files { + for f in &manifest.installed_files() { ensure!(valid_path(&f.path), "invalid manifest path"); ensure!( paths.insert(f.path.to_ascii_lowercase()), @@ -144,9 +184,75 @@ pub fn validate(manifest: &Manifest) -> Result<()> { ); } ensure!( - manifest.build_hash == build_hash(&manifest.files), + manifest.build_hash == build_hash(&manifest.installed_files()), "manifest buildHash mismatch" ); + if let Some(release) = &manifest.release { + ensure!( + release == &manifest.release_id(), + "invalid release identifier" + ); + } + if let Some(bundle) = &manifest.bundle { + ensure!( + manifest.platform.as_deref() == Some("macos-arm64"), + "bundle requires macos-arm64" + ); + ensure!( + bundle.archive.path == "MoonWell.app.tar.gz" + && bundle.archive.size > 0 + && bundle.archive.size <= 1024 * 1024 * 1024, + "invalid bundle archive" + ); + ensure!( + bundle.archive.sha256.len() == 64 + && bundle + .archive + .sha256 + .bytes() + .all(|b| b.is_ascii_hexdigit() && !b.is_ascii_uppercase()), + "invalid archive hash" + ); + ensure!( + manifest + .files + .iter() + .all(|f| f.path.starts_with("Data/") || f.path == "realmlist.wtf"), + "unexpected shared path" + ); + ensure!( + bundle.files.len() <= 10000 + && bundle + .files + .iter() + .try_fold(0_u64, |sum, f| sum.checked_add(f.size)) + .is_some_and(|sum| sum <= 2 * 1024 * 1024 * 1024), + "oversized bundle" + ); + ensure!( + bundle + .files + .iter() + .all(|f| f.path.starts_with("MoonWell.app/")), + "invalid bundle member" + ); + for path in [ + "MoonWell.app/Contents/MacOS/MoonWell", + "MoonWell.app/Contents/MacOS/launch", + ] { + ensure!( + bundle.files.iter().any(|f| f.path == path && f.executable), + "bundle executable missing" + ); + } + ensure!( + bundle + .files + .iter() + .any(|f| f.path == "MoonWell.app/Contents/Info.plist"), + "bundle plist missing" + ); + } Ok(()) } @@ -172,6 +278,33 @@ pub fn write(manifest: &Manifest, output: &Path) -> Result<()> { Ok(()) } +/// Select only common client files, preserving immutable object references. +pub fn shared_files(manifest: &Manifest, prefix: &str) -> Result> { + validate(manifest)?; + let files: Vec<_> = manifest + .files + .iter() + .filter(|f| f.path.starts_with("Data/") || f.path == "realmlist.wtf") + .cloned() + .collect(); + ensure!( + files.iter().any(|f| f.path == "realmlist.wtf") + && files.iter().any(|f| f.path.starts_with("Data/")), + "shared inventory is incomplete" + ); + for file in &files { + let key = file + .object_key + .as_deref() + .context("shared file has no immutable objectKey")?; + ensure!( + valid_path(key) && key.starts_with(&format!("{prefix}/releases/")), + "shared object is outside the release prefix" + ); + } + Ok(files) +} + #[cfg(test)] mod tests { use super::*; @@ -183,6 +316,7 @@ mod tests { size: 3, sha256: "abc".into(), object_key: None, + executable: false, }; assert_eq!( build_hash(&[f]), diff --git a/crates/moonwell-deploy/src/package.rs b/crates/moonwell-deploy/src/package.rs index 76a7c8886..6ccad7c73 100644 --- a/crates/moonwell-deploy/src/package.rs +++ b/crates/moonwell-deploy/src/package.rs @@ -393,6 +393,10 @@ pub fn validate_distribution(root: &Path) -> Result<()> { .contains(&u32::from_be_bytes(magic)), "MoonWell binary is not a native executable" ); + validate_shared_distribution(root) +} + +pub fn validate_shared_distribution(root: &Path) -> Result<()> { validate_data(&root.join("Data"))?; russian_archives(&root.join("Data"))?; agreement_files(&root.join("Data"))?; @@ -409,6 +413,20 @@ pub fn stage( output: &Path, source: Option<&Path>, host: &str, +) -> Result<()> { + stage_inner(Some(binary), data, output, source, host) +} + +pub fn stage_data(data: &Path, output: &Path, source: Option<&Path>, host: &str) -> Result<()> { + stage_inner(None, data, output, source, host) +} + +fn stage_inner( + binary: Option<&Path>, + data: &Path, + output: &Path, + source: Option<&Path>, + host: &str, ) -> Result<()> { let host = host.trim(); ensure!( @@ -423,7 +441,9 @@ pub fn stage( // A locale directory may point outside Data; it remains a read-only input as well. ensure_outside(archive.parent().context("language pack directory")?, output)?; } - ensure_outside(binary, output)?; + if let Some(binary) = binary { + ensure_outside(binary, output)?; + } for (_, path) in &agreements { ensure_outside(path, output)?; } @@ -434,31 +454,33 @@ pub fn stage( !output.exists(), "staging output already exists; choose a new directory" ); - let name = binary.file_name().context("binary filename is missing")?; - ensure!( - matches!( - name.to_str(), - Some("MoonWell.exe" | "MoonWell" | "MoonWell.app") - ), - "use a MoonWell player executable or .app" - ); fs::create_dir_all(output)?; - if binary.is_dir() { - for entry in WalkDir::new(binary).follow_links(false) { - let entry = entry?; - ensure!( - !entry.path_is_symlink(), - "bundle symlinks are not supported" - ); - let dest = output.join(name).join(entry.path().strip_prefix(binary)?); - if entry.file_type().is_dir() { - fs::create_dir_all(dest)?; - } else { - fs::copy(entry.path(), dest)?; + if let Some(binary) = binary { + let name = binary.file_name().context("binary filename is missing")?; + ensure!( + matches!( + name.to_str(), + Some("MoonWell.exe" | "MoonWell" | "MoonWell.app") + ), + "use a MoonWell player executable or .app" + ); + if binary.is_dir() { + for entry in WalkDir::new(binary).follow_links(false) { + let entry = entry?; + ensure!( + !entry.path_is_symlink(), + "bundle symlinks are not supported" + ); + let dest = output.join(name).join(entry.path().strip_prefix(binary)?); + if entry.file_type().is_dir() { + fs::create_dir_all(dest)?; + } else { + fs::copy(entry.path(), dest)?; + } } + } else { + fs::copy(binary, output.join(name))?; } - } else { - fs::copy(binary, output.join(name))?; } let target_data = output.join("Data"); fs::create_dir(&target_data)?; @@ -498,7 +520,11 @@ pub fn stage( output.join("realmlist.wtf"), format!("set realmlist {host}\n"), )?; - validate_distribution(output)?; + if binary.is_some() { + validate_distribution(output)?; + } else { + validate_shared_distribution(output)?; + } println!("Staged {}", output.display()); Ok(()) } diff --git a/crates/moonwell-deploy/src/storage.rs b/crates/moonwell-deploy/src/storage.rs index 21a36697c..fe4c6abad 100644 --- a/crates/moonwell-deploy/src/storage.rs +++ b/crates/moonwell-deploy/src/storage.rs @@ -14,6 +14,11 @@ pub trait Store { fn matches(&self, _key: &str, _file: &Path) -> Result { Ok(false) } + fn exists_with_size(&self, key: &str, size: u64) -> Result { + Ok(self + .get(key)? + .is_some_and(|bytes| bytes.len() as u64 == size)) + } fn upload(&self, key: &str, file: &Path) -> Result<()>; fn put_manifest(&self, key: &str, data: &[u8]) -> Result<()>; } @@ -323,6 +328,18 @@ impl Store for S3Store { } } + fn exists_with_size(&self, key: &str, size: u64) -> Result { + let (head, status) = self + .bucket + .head_object(key) + .map_err(|_| anyhow::anyhow!("S3 HEAD failed for {key}"))?; + ensure!( + status == 200 || status == 404, + "S3 HEAD {key}: HTTP {status}" + ); + Ok(status == 200 && head.content_length == i64::try_from(size).ok()) + } + fn upload(&self, key: &str, file: &Path) -> Result<()> { let (size, etag) = file_etag(file)?; if self.object_matches(key, size, &etag)? { @@ -373,9 +390,22 @@ pub fn publish( prefix != "World of Warcraft" && manifest_key != "manifest.json", "the legacy 3.3.5 publication keys are reserved" ); - let release = format!("{prefix}/releases/{}", m.build_hash); + if m.platform.is_none() { + m.platform = Some("windows".into()); + } + m.release = Some(m.release_id()); + let release = format!( + "{prefix}/releases/{}", + m.release.as_deref().expect("release identity") + ); + ensure!( + manifest_key != format!("{release}/manifest.json"), + "active key must not be the immutable inventory key" + ); for file in &mut m.files { - file.object_key = Some(format!("{release}/{}", file.path)); + if dir.join(&file.path).is_file() { + file.object_key = Some(format!("{release}/{}", file.path)); + } } println!( "Release: {release}; {} files; active manifest: {manifest_key}", @@ -392,6 +422,23 @@ pub fn publish( }; for file in &mut m.files { let path = dir.join(&file.path); + if !path.is_file() { + let key = file + .object_key + .as_deref() + .context("missing local file and shared reference")?; + ensure!( + manifest::valid_path(key) && key.starts_with(&format!("{prefix}/releases/")), + "invalid shared object key" + ); + ensure!( + store.exists_with_size(key, file.size)?, + "shared object is missing or has a different size: {}", + file.path + ); + println!("Referenced {}", file.path); + continue; + } let (size, hash) = manifest::hash_file(&path)?; ensure!( size == file.size && hash == file.sha256, @@ -432,11 +479,32 @@ pub fn publish( size ); } + if let Some(bundle) = &mut m.bundle { + let path = dir.join(&bundle.archive.path); + let (size, hash) = manifest::hash_file(&path)?; + ensure!( + size == bundle.archive.size && hash == bundle.archive.sha256, + "bundle archive changed after packaging" + ); + let key = format!("{release}/{}", bundle.archive.path); + bundle.archive.object_key = Some(key.clone()); + if !dry_run { + store.upload(&key, &path)?; + } + println!( + "{} {}", + if dry_run { "Would upload" } else { "Uploaded" }, + bundle.archive.path + ); + } if dry_run { return Ok(()); } // A concurrent editor or build must not produce a manifest describing different bytes. - for file in &m.files { + for file in m.files.iter().chain(m.bundle.iter().map(|b| &b.archive)) { + if !dir.join(&file.path).is_file() { + continue; + } let (size, hash) = manifest::hash_file(&dir.join(&file.path))?; ensure!( size == file.size && hash == file.sha256, @@ -626,4 +694,128 @@ mod tests { } } } + fn mac_fixture(dir: &Path) -> Manifest { + use crate::manifest::{ClientBundle, ClientFile}; + std::fs::write(dir.join("MoonWell.app.tar.gz"), b"archive").unwrap(); + let file = |path: &str, executable: bool| ClientFile { + path: path.into(), + size: 4, + sha256: format!("{:x}", sha2::Sha256::digest(b"stub")), + object_key: None, + executable, + }; + use sha2::Digest; + let mut data = file("Data/dbc.MPQ", false); + data.object_key = Some("MoonWell/releases/common/Data/dbc.MPQ".into()); + let mut realmlist = file("realmlist.wtf", false); + realmlist.object_key = Some("MoonWell/releases/common/realmlist.wtf".into()); + let (size, sha256) = manifest::hash_file(&dir.join("MoonWell.app.tar.gz")).unwrap(); + let bundle = ClientBundle { + archive: ClientFile { + path: "MoonWell.app.tar.gz".into(), + size, + sha256, + object_key: None, + executable: false, + }, + files: vec![ + file("MoonWell.app/Contents/MacOS/launch", true), + file("MoonWell.app/Contents/MacOS/MoonWell", true), + file("MoonWell.app/Contents/Info.plist", false), + ], + }; + let mut m = Manifest { + files: vec![data, realmlist], + bundle: Some(bundle), + platform: Some("macos-arm64".into()), + release: None, + build_hash: String::new(), + }; + m.build_hash = manifest::build_hash(&m.installed_files()); + m + } + + #[derive(Default)] + struct CompositionStore { + uploads: RefCell>, + inventories: RefCell>, + missing_data: bool, + fail_archive: bool, + } + impl Store for CompositionStore { + fn get(&self, _: &str) -> Result>> { + Ok(None) + } + fn exists_with_size(&self, key: &str, size: u64) -> Result { + assert!(key.starts_with("MoonWell/releases/common/")); + assert_eq!(size, 4); + Ok(!self.missing_data) + } + fn upload(&self, key: &str, _: &Path) -> Result<()> { + ensure!(!self.fail_archive, "simulated archive failure"); + self.uploads.borrow_mut().push(key.into()); + Ok(()) + } + fn put_manifest(&self, key: &str, bytes: &[u8]) -> Result<()> { + self.inventories + .borrow_mut() + .push((key.into(), serde_json::from_slice(bytes)?)); + Ok(()) + } + } + + #[test] + fn mac_publication_references_common_data_and_uploads_only_the_bundle() { + let dir = tempfile::tempdir().unwrap(); + let m = mac_fixture(dir.path()); + let store = CompositionStore::default(); + publish(&store, dir.path(), m.clone(), "MoonWell", "mac.json", false).unwrap(); + let uploads = store.uploads.borrow(); + assert_eq!(uploads.len(), 1); + assert!(uploads[0].ends_with("/MoonWell.app.tar.gz")); + let inventories = store.inventories.borrow(); + assert_eq!(inventories.len(), 2); + let (key, published) = &inventories[1]; + assert_eq!(key, "mac.json"); + assert_eq!( + published.files[0].object_key.as_deref(), + Some("MoonWell/releases/common/Data/dbc.MPQ") + ); + assert_eq!(published.release.as_deref(), Some(m.release_id().as_str())); + manifest::validate(published).unwrap(); + } + + #[test] + fn mac_dry_run_and_failed_publications_leave_active_inventory_unchanged() { + let dir = tempfile::tempdir().unwrap(); + let m = mac_fixture(dir.path()); + let dry = CompositionStore::default(); + publish(&dry, dir.path(), m.clone(), "MoonWell", "mac.json", true).unwrap(); + assert!(dry.uploads.borrow().is_empty() && dry.inventories.borrow().is_empty()); + for store in [ + CompositionStore { + missing_data: true, + ..Default::default() + }, + CompositionStore { + fail_archive: true, + ..Default::default() + }, + ] { + assert!(publish(&store, dir.path(), m.clone(), "MoonWell", "mac.json", false).is_err()); + assert!(store.inventories.borrow().is_empty()); + } + } + + #[test] + fn mac_inventory_rejects_unsafe_members_and_missing_executables() { + let dir = tempfile::tempdir().unwrap(); + let mut m = mac_fixture(dir.path()); + m.bundle.as_mut().unwrap().files[0].path = "MoonWell.app/../escape".into(); + m.build_hash = manifest::build_hash(&m.installed_files()); + assert!(manifest::validate(&m).is_err()); + let mut m = mac_fixture(dir.path()); + m.bundle.as_mut().unwrap().files[0].executable = false; + assert!(manifest::validate(&m).is_err()); + } } diff --git a/crates/moonwell/README.md b/crates/moonwell/README.md index 95dab78cc..463143257 100644 --- a/crates/moonwell/README.md +++ b/crates/moonwell/README.md @@ -34,10 +34,17 @@ python3 crates/moonwell/package_macos.py \ This creates `target/release/bundle/MoonWell.app` with the MoonWell name and app icon. Packaging requires a macOS executable and an ICNS icon; it does not convert a -Windows EXE into a Mac application. Build natively on each target Mac architecture. +Windows EXE into a Mac application. Only Apple Silicon (`arm64`) player binaries are accepted. Pass `--output` to choose a different destination and `--bundle-id` to set your application identifier. An existing destination is never overwritten. +The packager compiles a native `launch` entry point and applies free ad hoc signing +to the nested code and app bundle. Python packaging requires Xcode command-line +tools; no paid signing identity is used. macOS may require explicit security approval. + +For archive creation and publication with shared data, use +[`deploy-moonwell-macos.py`](../../scripts/deploy-moonwell-macos.py). + Launch the `.app` from Finder. Its launcher finds your own `Data/` or `WoW/Data/` beside `MoonWell.app` and keeps `benilla-config/` there too, outside the bundle. Explicit `WOW_DATA` and `BENILLA_HOME` values take priority. diff --git a/crates/moonwell/launch_macos.c b/crates/moonwell/launch_macos.c new file mode 100644 index 000000000..0328da176 --- /dev/null +++ b/crates/moonwell/launch_macos.c @@ -0,0 +1,45 @@ +// Native bundle entry point: keep writable state and game data outside the app. +#include +#include +#include +#include +#include +#include +#include +#include + +int main(int argc, char **argv) { + (void)argc; + char executable[PATH_MAX], resolved[PATH_MAX], macos[PATH_MAX], root[PATH_MAX]; + char game[PATH_MAX], state[PATH_MAX], data[PATH_MAX]; + uint32_t length = sizeof(executable); + if (_NSGetExecutablePath(executable, &length) != 0 || !realpath(executable, resolved)) return 1; + char *slash = strrchr(resolved, '/'); + if (!slash) return 1; + *slash = '\0'; + if (snprintf(macos, sizeof(macos), "%s", resolved) >= (int)sizeof(macos)) return 1; + for (int i = 0; i < 3; ++i) { + slash = strrchr(resolved, '/'); + if (!slash) return 1; + *slash = '\0'; + } + if (snprintf(root, sizeof(root), "%s", resolved) >= (int)sizeof(root) || + snprintf(game, sizeof(game), "%s/MoonWell", macos) >= (int)sizeof(game) || + snprintf(state, sizeof(state), "%s/benilla-config", root) >= (int)sizeof(state)) return 1; + if (!getenv("BENILLA_HOME") && setenv("BENILLA_HOME", state, 1) != 0) return 1; + if (!getenv("WOW_DATA")) { + struct stat info; + const char *folders[] = {"Data", "WoW/Data"}; + for (int i = 0; i < 2; ++i) { + if (snprintf(data, sizeof(data), "%s/%s", root, folders[i]) >= (int)sizeof(data)) return 1; + if (stat(data, &info) == 0 && S_ISDIR(info.st_mode)) { + if (setenv("WOW_DATA", data, 1) != 0) return 1; + break; + } + } + } + argv[0] = game; + execv(game, argv); + perror("Cannot start MoonWell"); + return 1; +} diff --git a/crates/moonwell/package_macos.py b/crates/moonwell/package_macos.py index ae1b39a90..d370f174e 100644 --- a/crates/moonwell/package_macos.py +++ b/crates/moonwell/package_macos.py @@ -5,30 +5,14 @@ import argparse import os from pathlib import Path import plistlib +import re import shutil import struct +import subprocess +import sys import tomllib -# Keep the install and writable client state beside the app, outside the bundle. -LAUNCHER = '''#!/bin/sh -set -eu -macos_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd) -player_dir=$(CDPATH= cd -- "$macos_dir/../../.." && pwd) -if [ "${BENILLA_HOME+x}" != x ]; then - export BENILLA_HOME="$player_dir/benilla-config" -fi -if [ "${WOW_DATA+x}" != x ]; then - for data_dir in "$player_dir/Data" "$player_dir/WoW/Data"; do - if [ -d "$data_dir" ]; then - export WOW_DATA="$data_dir" - break - fi - done -fi -exec "$macos_dir/MoonWell" "$@" -''' - def main(): parser = argparse.ArgumentParser(description=__doc__) @@ -38,6 +22,16 @@ def main(): parser.add_argument("--output", type=Path, default=Path("target/release/bundle/MoonWell.app")) parser.add_argument("--bundle-id", default="org.moonwell.client") args = parser.parse_args() + if sys.platform != "darwin": + parser.error("packaging requires macOS") + if subprocess.check_output(["lipo", "-archs", str(args.binary)], text=True).strip() != "arm64": + parser.error("--binary must target Apple Silicon only") + + load_commands = subprocess.check_output(["otool", "-l", str(args.binary)], text=True) + minimum_match = re.search(r"\bminos\s+([0-9.]+)", load_commands) + if minimum_match is None: + parser.error("--binary has no macOS minimum deployment version") + minimum_macos = minimum_match.group(1) with args.binary.open("rb") as binary: magic = binary.read(4) @@ -66,7 +60,9 @@ def main(): resources.mkdir() shutil.copyfile(args.binary, macos / "MoonWell") (macos / "MoonWell").chmod(0o755) - (macos / "launch").write_text(LAUNCHER, encoding="utf-8", newline="\n") + subprocess.run(["clang", "-arch", "arm64", "-O2", "-Wall", "-Wextra", "-Werror", + f"-mmacosx-version-min={minimum_macos}", + str(Path(__file__).with_name("launch_macos.c")), "-o", str(macos / "launch")], check=True) (macos / "launch").chmod(0o755) shutil.copyfile(args.icon, resources / "MoonWell.icns") with (contents / "Info.plist").open("wb") as plist: @@ -81,7 +77,12 @@ def main(): "CFBundleShortVersionString": version, "CFBundleVersion": version, "NSHighResolutionCapable": True, + "LSMinimumSystemVersion": minimum_macos, }, plist) + # Sign nested native code first, then seal the whole bundle. No paid identity is used. + for path in [macos / "MoonWell", macos / "launch", args.output]: + subprocess.run(["codesign", "--force", "--sign", "-", "--timestamp=none", str(path)], check=True) + subprocess.run(["codesign", "--verify", "--deep", "--strict", str(args.output)], check=True) print(os.fspath(args.output.resolve())) diff --git a/scripts/deploy-moonwell-macos.py b/scripts/deploy-moonwell-macos.py new file mode 100644 index 000000000..975a677d3 --- /dev/null +++ b/scripts/deploy-moonwell-macos.py @@ -0,0 +1,103 @@ +#!/usr/bin/env python3 +"""Build, package and optionally publish the Apple Silicon game with shared data.""" +import argparse +import gzip +import hashlib +import json +import os +from pathlib import Path +import stat +import subprocess +import sys +import tarfile +from datetime import datetime, timezone + + +def hash_file(path): + with path.open('rb') as stream: + return hashlib.file_digest(stream, 'sha256').hexdigest() + + +def archive_bundle(bundle, output): + """Write deterministic USTAR regular entries only, matching the launcher's reader.""" + files = [] + paths = sorted(bundle.rglob('*')) + for path in paths: + if path.is_symlink() or not (path.is_dir() or path.is_file()): + raise ValueError(f'Unsupported bundle entry: {path}') + if not path.is_file(): + continue + relative = path.relative_to(bundle.parent).as_posix() + if any(c in relative for c in '\\:\r\n\0'): + raise ValueError(f'Invalid bundle filename: {relative}') + files.append({'path': relative, 'size': path.stat().st_size, + 'sha256': hash_file(path), 'executable': bool(path.stat().st_mode & 0o111)}) + if len(files) > 10000 or sum(f['size'] for f in files) > 2 * 1024**3: + raise ValueError('Bundle exceeds launcher extraction limits') + archive = output / 'MoonWell.app.tar.gz' + with archive.open('xb') as raw: + with gzip.GzipFile(filename='', fileobj=raw, mode='wb', mtime=0) as compressed: + with tarfile.open(fileobj=compressed, mode='w', format=tarfile.USTAR_FORMAT) as tar: + for file in files: + entry = tarfile.TarInfo(file['path']) + entry.size = file['size'] + entry.mode = 0o755 if file['executable'] else 0o644 + with (bundle.parent / file['path']).open('rb') as stream: + tar.addfile(entry, stream) + if archive.stat().st_size > 1024**3: + raise ValueError('Bundle archive exceeds launcher download limit') + descriptor = {'path': archive.name, 'size': archive.stat().st_size, + 'sha256': hash_file(archive), 'files': files} + (output / 'bundle.json').write_text(json.dumps(descriptor, indent=2) + '\n') + return descriptor + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--icon', type=Path, required=True) + parser.add_argument('--binary', type=Path, help='Use an existing arm64 player build') + parser.add_argument('--output', type=Path) + parser.add_argument('--env-file', type=Path) + parser.add_argument('--shared-manifest-key', default='manifest-1.12.1-data.json') + parser.add_argument('--manifest-key', default='manifest-1.12.1-macos-arm64.json') + parser.add_argument('--prefix', default='World of Warcraft 1.12.1') + mode = parser.add_mutually_exclusive_group() + mode.add_argument('--publish', action='store_true') + mode.add_argument('--dry-run', action='store_true') + args = parser.parse_args() + if sys.platform != 'darwin': + parser.error('Build and package on an Apple Silicon Mac') + if args.manifest_key in ('manifest.json', 'manifest-1.12.1.json', args.shared_manifest_key): + parser.error('Mac manifest key must be separate from Windows and shared data') + if (args.publish or args.dry_run) and not args.env_file: + parser.error('--env-file is required for publication and dry-run') + root = Path(__file__).resolve().parents[1] + output = (args.output or root / 'dist' / ('moonwell-macos-' + datetime.now(timezone.utc).strftime('%Y%m%d-%H%M%S'))).resolve() + output.mkdir(parents=True, exist_ok=False) + if not args.binary: + subprocess.run(['cargo', 'build', '-p', 'moonwell', '--release', '--no-default-features', '--locked', + '--target', 'aarch64-apple-darwin'], cwd=root, check=True) + binary = (args.binary or root / 'target/aarch64-apple-darwin/release/MoonWell').resolve() + bundle = output / 'build' / 'MoonWell.app' + subprocess.run([sys.executable, str(root / 'crates/moonwell/package_macos.py'), + '--binary', str(binary), '--icon', str(args.icon.resolve()), '--output', str(bundle)], check=True) + # A non-system library must be bundled and signed deliberately, not assumed present on player Macs. + dependencies = subprocess.check_output(['otool', '-L', str(bundle / 'Contents/MacOS/MoonWell')], text=True).splitlines()[1:] + for line in dependencies: + library = line.strip().split(' (', 1)[0] + if not library.startswith(('/System/Library/', '/usr/lib/')): + raise RuntimeError(f'Unbundled runtime dependency: {library}') + archive_bundle(bundle, output) + if args.publish or args.dry_run: + command = ['cargo', 'run', '-p', 'moonwell-deploy', '--release', '--locked', '--', 'publish-macos', + '--dir', str(output), '--env-file', str(args.env_file.resolve()), + '--shared-manifest-key', args.shared_manifest_key, '--manifest-key', args.manifest_key, + '--prefix', args.prefix] + if args.dry_run: + command.append('--dry-run') + subprocess.run(command, cwd=root, check=True) + print(f'Mac package: {output}') + + +if __name__ == '__main__': + main()