Using photometric data from the Sloan Digital Sky Survey (SDSS) Release 17, this study builds a modular system to examine and categorize stellar populations. The framework effectively identifies main sequence stars, giants, and white dwarfs by combining machine learning techniques such as K-Means, DBSCAN, random forests, and support vector machines (SVM). Advanced clustering and classification techniques, backed by dimensionality reduction techniques like Principal Component Analysis (PCA), match theoretical data with stellar evolutionary tracks. The results show that the methods used to classify stellar populations, show Galactic substructures, and model evolutionary paths are very accurate and reliable. With the flexible framework, artificial intelligence and data science can be used to make astrophysics study better. In the future, bigger datasets and smarter algorithms will be used to make galactic exploration even better.

Github Jupyter Notebook

AJupyter Notebook companion for the complete project at SDSS_Release_17_system_model/sdss_data.ipynb at main · Fr4nc3/SDSS_Release_17_system_model

1. Introduction

The Sloan Digital Sky Survey (SDSS) is one of the biggest and most important astronomy studies ever (York et al., 2000). It has changed the way we think about the universe. Since its start in 2000, SDSS has taken high-resolution pictures and measured the spectra of millions of stellar objects, making a huge dataset that anyone can use (Abazajian et al., 2009). With each new set of data, SDSS has improved its skills, giving us more information about how galaxies, stars, and the universe itself are put together and how they change over time (Ivezić et al., 2008). As part of its huge collection of data, the survey’s photometric system provides accurate readings in five bands (u, g, r, i, z) that are needed to sort and study heavenly objects (Fukugita et al., 1996). The photometric dataset from SDSS is used in this project. It focuses on stars that meet high-quality standards for clean and accurate data (Abazajian et al., 2009). The dataset used in this project is made up of 5,000 stars with clean photometry that were chosen from the SDSS process using strict criteria. There are photometric magnitudes in five bands (u, g, r, i, z), astronomical positions (RA, Dec), and photometric quality flags for each entry (Fukugita et al., 1996). The “clean photometry” label guarantees that the data is free of problems like swollen pixels, cosmic rays, and edge effects, which can cause problems in studies (Abazajian et al., 2009). Focusing on this group makes sure that the results are reliable and correct while keeping the dataset size reasonable for in-depth modeling and research. With its clear and high-quality images of stellar photometric features in a small area of the sky, the dataset is perfect for building a system model (Bovy et al., 2012). This clean photometric sample contains stellar populations, and the main objective of this project is to create a system model for a dataset. that studies and labels them. In order to find different stellar populations and model their connections in multi-dimensional photometric space, the project will use grouping and classification methods (Ball & Brunner, 2010). This includes making color-magnitude graphs, finding patterns in photometric features, and checking that the model works well at sorting stellar populations. In the end, the system model will be a strong framework for studying stellar datasets, showing how clean and organized data can be used to gain important insights into Galactic stellar populations (Ivezić et al., 2008).

2. Problem Statement

The SDSS photometric datasets provide a unique chance to examine Galactic stellar populations and Milky Way structure (Jurić et al., 2005; Gao et al., 2012). Based on exact measurements in five photometric bands, these datasets allow extensive stellar property study (Amado et al., 2017; An & Beers, 2020). The massive volume and high dimensionality of this data offer analysis and interpreting challenges (Chen et al., 2016). Astronomers and engineers struggle to get insights due to observational unpredictability, noise, and inconsistency (Eisenstein et al., 2007). The difficulty of differentiating stellar populations across many photometric dimensions makes it hard to find patterns and linkages needed to comprehend Galactic structure (Bovy et al., 2011). Current techniques are scattered and typically fail to meet this dataset’s multifaceted challenges (Gao et al., 2012). Integrating and classifying stellar populations is inefficient since there is no formal framework for integrating and evaluating clean photometry datasets (Chen et al., 2016). This gap hinders scientific discovery and future survey data processing workflow scalability (An et al., 2012). The genesis and evolution of stellar populations cannot be addressed without a reliable and systematic method (Jurić et al., 2005). For academics and engineers to build on one another’s work, the lack of such frameworks limits cooperation and standardization (Eisenstein et al., 2007). This lack of integration across datasets and approaches prevents deeper Galactic evolution insights (Chen et al., 2016). As astronomical data grows, these challenges become more urgent (Amado et al., 2017).

3. Experimental Approach

I. Data Collection and Preprocessing

The dataset of the investigation comprises 5,000 entries from the SDSS Release 17 photometric catalog (Sánchez et al., 2021). These entries were judged to offer pristine, high-quality data fit for modeling and analysis. In order to analyze Galactic stellar populations, we employed SQL to perform a tight filtering process that concentrated on key fields (Beck et al., 2016; Oyaizu et al., 2007). Review Appendix B for Python code for this section.

A. Selection

The dataset selection process retrieved photometric magnitudes in the u, g, r, i, and z bands, positional data such as RA and Dec, and photometric quality flags. The RA range was limited to 180°–181° and Dec -0.5⁰—0.5⁰ to focus on a specific region of the sky (Ivezić et al., 2004; Bramich et al., 2012). These constraints ensured the dataset’s relevance to the project’s objective of studying hierarchical galactic structures. The selected fields provide the foundation for deriving features like color indices and spatial distributions essential for clustering and classification (Vilella-Rojo et al., 2015).

B. Quality Filters

For data reliability, quality filters got rid of entries that had issues like noise, edge effects, and cosmic ray contamination. Retaining 2,536 high-quality entries (50.72% of the initial dataset), the photometric quality filtering process made use of photometric quality flags like EDGE and NOPROFILE and a PSF magnitude error threshold (Jin & Hirakawa, 2012; Zhang & Kainulainen, 2019). Maintaining statistical power and data accuracy are both balanced by this threshold. The retained dataset is good for deep analyses because it gets rid of bad entries without changing the types of stellar populations (Rhoads, 2000; Bramich & Freudling, 2012).

C. Filtering

Reliable photometric measurements were used to make sure that only stars that passed quality control steps were included. The final subset of the dataset retained precise measurements of magnitudes and spatial positions but excluded entries with photometric flaws that were flagged. PSF magnitude errors analysis showed that most bands had consistent values, with only a few outliers having a small effect on the quality of the data (Goessl & Riffeser, 2001; Van Dokkum, 2001). As part of the modular system’s reliability, the filtering process improved the dataset so that it could be used for clustering and classification later on (Smale et al., 2010).

D. Integration with System Model

The prepared dataset aligns with the modular system’s requirements, serving as the foundation for clustering and classification models. Positional and photometric data ensure the dataset supports the project’s goal of distinguishing in situ and accreted stellar populations. CMD-based validation and clustering metrics will further confirm the effectiveness of these preprocessing steps in later stages (Goessl & Riffeser, 2001; Zhang & Kainulainen, 2019).

E. Key Fields for Analysis

II. Workflow

A. Subset the Data

We selected photometric magnitudes in the u, g, r, i, z bands and positional data (RA, Dec) for mapping stellar traits. Quality flags filtered entries with issues like cosmic rays or edge effects. ensuring a clean dataset for further analysis (Desai et al., 2016; Rhoads, 2000). The dataset spans RA values from 180⁰ to 181⁰ and Dec values from -0.5⁰ to 0.5⁰, covering a defined spatial area (Goessl & Riffeser, 2001). This prepared subset provides robust data for exploring Galactic stellar populations and aligns with project objectives for accurate modeling and classification (Ivezić et al., 2004; Majaess et al., 2014). Review appendix C for Python code for this section.

B. Exploratory Data Analysis (EDA)

Visualization methods made the structure of the dataset stand out. For main-sequence stars, giants, and white dwarfs, CMDs showed brightness and color trends with distinct sequences (Souza et al., 2020). As an example, a CMD plot of g-r vs. r revealed theoretical clustering patterns that proved the continuity of the dataset (Pasquato & Milone, 2019). It was proven that the dataset had uniform sampling by a scatter plot of RA vs. Dec, which showed even spatial coverage and no unexpected clustering. When looking at color indices (g-r and r-i), they showed expected bimodal distributions, which made them ready for modeling (Souza et al., 2020; Pasquato & Milone, 2019). The dataset was checked to make sure it meets the needs for accurate modeling and classification.

C. Build the System Model

The system model integrates positional and photometric data to classify stellar populations. RA and Dec reveal spatial distributions, while photometric magnitudes and color indices capture brightness and color properties. Initial clustering methods, like K-Means, identify natural subpopulations, such as main-sequence stars and giants, while DBSCAN identifies complex, non-linear groupings (Hu et al., 2021; Ordovás-Pascual & Almeida, 2014). Clustering results feed supervised classification models, including Random Forest and SVM, to refine population categorization and detect potential new stellar types (Souza et al., 2020; Conroy & van Dokkum, 2016). PCA reduces dimensionality, optimizing computational efficiency while retaining key photometric features. The modular framework supports adaptation and scalability, aligning with the project’s goals of identifying hierarchical galactic structures and understanding stellar evolution (Collin et al., 2020; Goessl & Riffeser, 2001).

III. System Modeling Techniques

Our modular system model studies, categorizes, and ensures data reliability for stellar populations. This framework allows for flexible clustering and classification while maintaining data accuracy (Elorrieta et al., 2016; Collin et al., 2020).

A. Analyzing Stellar Populations

The system uses photometric magnitudes across the u, g, r, i, z bands to derive key features like color indices g-r, r-i, and u-g (Fukugita et al., 1996; Ivezić et al., 2008). With these indices, we can learn a lot about the brightness, temperature, and color of stars. Positional features like RA and Dec are combined with photometric data to map spatial distributions and find patterns in stellar populations (Elorrieta et al., 2016; Collin et al., 2020). Our understanding of galactic evolution is improved by the patterns and connections discovered by this research. The hierarchical structure of the galaxy and the motions of its stellar components can be seen by looking at the links between brightness, color, and spatial positioning (Horta et al., 2022; Das et al., 2019). In order to understand stellar distributions and evolutionary trajectories, this modular analysis ensures a solid framework (Bovy et al., 2011).

B. Framework for Classification

The modular system model relies on the classification framework to distinguish in situ and accreted stellar populations (Nag & Pal, 2016). Using photometric data such as magnitudes and color indices, the framework distinguishes between stellar types, including main-sequence stars, giants, and white dwarfs (Das et al., 2019; Lu et al., 2017). This classification goes right to modular structure, making galaxy hierarchical structures easier to understand. Identifying population distributions and evolutionary tendencies improves Galactic assembly understanding (Elorrieta et al., 2016). By integrating stellar results with clustering insights, the modular framework enhances the classification system’s ability to validate stellar groups and examine their astrophysical importance (Nag & Pal, 2016; Collin et al., 2020). This integration is necessary for the stellar system model to operate with the project’s mapping and understanding of stellar populations (Horta et al., 2022).

C. Data Reliability

The accuracy of clustering and classification results within the modular system is dependent on the data reliability (Mo & Siepel, 2023). The model that identified stellar populations reflects true astrophysical properties rather than observational artifacts by maintaining a dataset with consistent photometric measurements and minimizing noise (Zhu et al., 2006; Desai et al., 2016). Reliable data not only improves clustering’s predictive power but also confirms classifications, making sure that the results match up with known Galactic structures (Pasquato & Milone, 2019). The system is better able to provide insightful information about stellar development and Galactic assembly as a result of its emphasis on accuracy (Bovy et al., 2012). A solid pipeline that supports accurate modeling of the galaxy’s structure and past is created by the integration of reliability checks into the modular system model (Goessl & Riffeser, 2001).

IV. Validation Methods

In order to guarantee the accuracy and reliability of the system model, we use a combination of cross-validation, CMD-based clustering validation, and optional comparisons with known stellar structures.

A. Cross-Validation:

Cross-validation evaluates the system model’s ability to generalize to unseen data by iteratively training and testing on different dataset subsets (Bai et al., 2019). For clustering, silhouette scores and inter-cluster distances assess the cohesion and separation of identified groups, ensuring meaningful stellar subpopulations. For classification models, metrics such as accuracy, precision, recall, and F1-score validate the model’s performance (Gaia Collaboration et al., 2020; Castro-Ginard et al., 2020). This step ensures the modular framework produces consistent and reliable results across its components.

B. CMD-Based Clustering Validation

CMDs are necessary for validating clustering results visually and quantitatively. We can be sure that the matching of clusters with expected stellar evolutionary tracks is correct by plotting color indices against magnitudes. Metrics like intra-cluster variance and cluster centroid consistency show how reliable the results of clustering are. This validation makes sure that the modular system can accurately tell the difference between main-sequence stars, giants, and white dwarfs, which reflects the hierarchical Galactic structures (Borsato et al., 2019; Ou et al., 2022).

C. Comparison with Known Stellar Structures

Well-studied galactic substructures like Gaia-Enceladus and Sequoia are used to compare the system model’s results for clustering and classifying things. Validating the model’s ability to reproduce well-known patterns are the spatial and photometric features of these known substructures. This comparison also helps find new substructures, which improves the modular framework’s trustworthiness and helps us learn more about how Galactic structures are put together (Necib et al., 2019; Feuillet et al., 2020; Ou et al., 2022).

4. Results

We did an extensive analysis of the SDSS Release 17 dataset. We can review appendix D for Python code for this section.

I. Data Preparation and Descriptive Statistics

The dataset includes stellar photometric and spatial information. The dataset spans a small but well-defined sky region, as shown by the RA and Dec values, which range from 180.0005 to 180.9999 degrees (Cantat-Gaudin et al., 2018). This spatial limitation keeps the investigation on a constant stellar field, preventing contamination from distant or unrelated locations. Before filtering, the dataset had a 100% retention rate, suggesting that all items satisfied analytical quality norms (Stassun et al., 2019). The photometric magnitudes cross the u, g, r, i, and z bands and show a diverse range of stellar brightness levels. The median magnitudes for these bands are the r-band at 21.48 and the i-band at 20.86, indicating typical stars. Significant brightness fluctuation typical of varied stellar populations is seen in the standard deviations. Most photometric observations have reliable uncertainties, with 75th percentile errors < 0.64 in all bands except where errors are largest. Outliers, such as a 51.53 maximum error in the u-band, result from infrequent measurement issues but do not affect dataset reliability (Lindegren et al., 2021). Key validation indicators stress the dataset’s analytical readiness. Without any significant gaps or clustering abnormalities, the uniform distribution of RA and Dec suggests uniform sky coverage. The dataset is suitable for clustering and classification tasks because of the conformity of photometric magnitude distributions with theoretical assumptions. The dataset’s quality and reliability for analyzing Galactic stellar populations are confirmed by these observations and validations (Bovy et al., 2016).

A. Table 1: RA and Dec Ranges

B. Table 2: Photometric Magnitude Statistics

C. Table 3: PSF Magnitude Error Statistics

II. Exploratory Data Analysis (EDA)

A. Color-Magnitude Diagrams (CMDs)

CMDs serve as crucial tools to identify stellar sequences, validate stellar alignment with dataset clustering, and cluster different evolutionary stages.

i. Color-Magnitude Diagram (g-r vs. r)

The Color-Magnitude Diagram (CMD) shows a well-defined stellar sequence by plotting the (g−r) color index against the (r) magnitude, effectively highlighting stellar populations (Kalirai & Tosi, 2004). A thick and continuous main sequence, indicative of stars in their primary hydrogen-burning phase, is prominently visible (Nardiello et al., 2018). According to theoretical models of mature stellar populations, a branch above the main sequence may represent giant stars (Li et al., 2015). Having few outliers or deviations from predicted patterns demonstrates the dataset's reliability (Belloni et al., 2017). The CMD verifies that the dataset aligns well with theoretical stellar clustering, enabling detailed astronomical studies (Blum et al., 2006). It better illustrates stellar subgroups like main-sequence stars and giants, underscoring its value in researching Galactic stellar populations (Chilingarian & Zolotukhin, 2011). This visual evidence suggests the dataset’s robustness for clustering and classification studies (Cassisi, 2012). The absence of significant outliers enhances confidence in dataset quality and purity (Saito et al., 2012). By effectively capturing stellar evolutionary trajectories, the CMD proves essential for studying Galactic structure and evolution (Kalirai & Tosi, 2004). The robust visual framework it provides supports advanced clustering and classification techniques in astrophysics (Blum et al., 2006).

ii. Color-Magnitude Diagram (u-g vs. g)

Insights into hotter and bluer stellar populations are provided by the u-g vs. g color-magnitude map. The plot shows a focused main sequence with stars extending across a range of u-g values, representing their various temperatures and ages (Chilingarian & Zolotukhin, 2011). Higher u-g values, which show cooler stars, are linked to hotter and younger stellar populations, while lower u-g values show bluer stars (Baraffe et al., 1998). As expected, the clear split and gradual spread of stars in the dataset support the evolutionary tracks for blue and ultraviolet-bright stars (Blakeslee et al., 2003). This CMD shows that the dataset can look into the hotter, earlier stages of stellar evolution (Kalirai & Tosi, 2004). This makes it a useful tool for studying groups of young, active stars (Savino et al., 2023).

iii. Color-Magnitude Diagram (r-i vs. i)

We can see important trends in stellar populations by plotting the CMD for the r-i color index against the i-band magnitude (Kalirai & Tosi, 2004). The picture shows a clear main sequence with a wide, steady distribution at smaller r-i values, which means the stars are in their main hydrogen-burning phase (Nardiello et al., 2018). The presence of giants and other developed stellar groups may be indicated by a separation that appears as r-i values rise (Li et al., 2015). The tight clustering and steady spread of stars in this dataset demonstrate the model's reliability and agreement with theoretical models of stellar evolution (Belloni et al., 2017). The low number of outliers in the dataset shows that it is of high quality, making it better for further clustering and classification studies (Blum et al., 2006). Furthermore, the CMD shows how useful r-i is as a color index for separating different evolutionary stages and subpopulations in the Galactic stellar assembly (Chilingarian & Zolotukhin, 2011). Validating and understanding the outcomes of the clustering and classification processes will be made possible by this plot (Cassisi, 2012). The observed trends align with prior research on CMDs, confirming the dataset's capacity to support astrophysical investigations (Saito et al., 2012).

iv. Color-Magnitude Diagram (i-z vs. z)

In the i-z vs. z color-magnitude image, cooler stars in the later stages of evolution are shown alongside fainter and redder stellar groups (López-Cruz et al., 2004). The plot shows a main sequence that is tightly grouped and has little spread. This shows how accurate the dataset is at measuring redder color indices (Cassata et al., 2007). There is a smaller range of i-z values, which means that the stars are all cooler. These stars include giants and advanced stellar types (Jaffé et al., 2010). This CMD is very helpful for finding late-type stars and studying the red end of the Galactic stellar population. The plot shows that the dataset is good for studying fainter, cooler stellar objects, giving us information about the later stages of stars' lives.

B. Histograms of Magnitudes

i. Distribution of Magnitudes

a. Distribution of psfMag_u

The psfMag_u graph shows a range of magnitudes that are evenly spread out, with the brightest points being around 23 and 24 magnitudes. This is about the brightness level that is predicted for stars in the ultraviolet (u-band) (Thuan & Gunn, 1976). This band is mostly made up of the properties of younger, hotter stellar populations that give off a lot of ultraviolet light (Markov et al., 2001). A wide spectrum of stellar types is represented by the slow rise and fall on either side of the peak, which shows a balance between larger and fainter stars (Lardo et al., 2010). Significantly reducing the possibility of observational biases or reduction mistakes that could mess up the analysis is the lack of sudden cutoffs or outliers. Since it enables a precise measurement of their population patterns and evolutionary stages, this uniformity is essential for studying ultraviolet-bright stars (Christlein et al., 2004). The u-band is useful for studying young, high-energy stellar populations because it can pick up a wide range of stellar properties (Sullivan et al., 2003).

b. Distribution of psfMag_g

The histogram for psfMag_g shows a distribution that peaks around 22 to 24 magnitudes, indicating the most common brightness levels for stars in the g-band (Danielski et al., 2018). Compared to psfMag_u, the g-band covers slightly cooler stars, capturing a wider range of stellar temperatures and filling the gap between the u-band's bright ultraviolet stars and the r-band's mid-UV stellar populations (Lardo et al., 2010). A balanced dataset, capturing a wide range of stellar brightness levels, is shown by the gradual increase in frequency up to the peak and the symmetric decline on either side (Kron & Roach, 1988). The fact that the g-band photometry accurately depicts a range of stellar populations is supported by the smooth distribution and lack of sharp cuts (Buchholz et al., 2009). Insightful information about the dataset's mid-range brightness is given by this histogram, which also shows how important the g-band is for cross-wavelength photometric analysis (Kuntzer et al., 2016).

c. Distribution of psfMag_r

The majority of stars in the r-band exhibit this brightness level, as shown by the sharp peak near 22 magnitudes in the histogram for psfMag_r (Buchholz et al., 2009). The peak's sharpness emphasizes the predominance of mid-brightness stars, which shows that the dataset was designed with capturing this main population in mind. This pattern shows a slow rise from brighter magnitudes (lower values) and a faster fall after the peak, which emphasizes the concentration of stars in the middle brightness range (Kron & Roach, 1988). Fewer brighter and fainter outliers show that the r-band photometric measurements are accurate, and the dataset accurately shows typical stellar populations (Danielski et al., 2018). The r-band is essential for thorough stellar studies because it acts as a crucial midpoint between the bluer g-band and redder wavelengths like i and z in photometric analyses.

d. Distribution of psfMag_i

The histogram for psfMag_i shows a relatively symmetric distribution centered around a peak near 21 magnitudes, highlighting the dataset’s ability to capture evolved stellar populations, including giants (Lardo et al., 2010). This balanced distribution reflects the dataset’s coverage of stars transitioning through later evolutionary stages, consistent with the role of the i-band in identifying mid-to-late stellar phases (Buchholz et al., 2009). The gradual rise from brighter magnitudes (lower values) and a balanced decline toward fainter magnitudes indicate a well-sampled dataset with minimal bias, ensuring robust representation across the stellar population (Kron & Roach, 1988). The histogram also complements the CMDs, where the i-band contributes significantly to distinguishing giants and other evolved groups, emphasizing its critical role in identifying late-evolutionary stages and informing clustering and classification analyses (Kuntzer et al., 2016).

e. Distribution of psfMag_z

The z-band's significance in capturing cooler, redder stars and evolved stellar groups is highlighted by the histogram for psfMag_z, which shows a distribution that is centered around a peak near 21 magnitudes. It's important to use uniform sampling to study fainter stellar populations near the brightness limit of the dataset, which is shown by the steady rise from brighter magnitudes and fall toward fainter magnitudes. This even distribution makes sure that the zz-band can effectively pick up a wide range of stellar properties, even those of stars that are in advanced evolutionary stages. This strengthens its function in studying redder wavelengths (Buchholz et al., 2009). The dataset is even more reliable because it doesn't have any sudden cutoffs. This makes it possible to do a thorough analysis of late-evolutionary stages and cooler stellar populations (Kuntzer et al., 2016).

C. Color Index Distributions

i. Distribution of Color Indices:

a. Histogram for g-r

It is clear that the g-r color index is bimodal, with peaks on the histogram at about 0.9 and 1.8 (Milone et al., 2016; Lee, 2017). These peaks show different kinds of stellar peaks in the dataset (Bellini et al., 2009). They are made up of main-sequence stars and giant stars. The distinct separation between the modes shows that the dataset can divide subgroups by temperature and color (Chilingarian & Zolotukhin, 2011). The fact that there is an even spread within each peak shows that the samples were taken in the same way every time, which lowers observational bias. The wide range of values, from -0.98 to 5.15, further demonstrates the dataset's capacity to capture a variety of stellar properties, including rare or unusual types of stars (Chang et al., 2012). This dataset has a lot of different things in it, which makes it great for tasks like classification and clustering.

b. Histogram for r-i

The r-i color index histogram has a narrower and more concentrated distribution centered around 0.4, which shows that cooler stars and late-type stellar populations are more common (Milone et al., 2016). There is some variation in the range from -3.67 to 2.93, but most of the values are close together in the interquartile range of 0.14 to 0.79 (Lee, 2017). This small range makes sure that there isn't much overlap between stellar subgroups, which makes classification tasks more reliable (Bellini et al., 2009). The high number of values near the middle of the dataset helps it effectively group stars with similar evolutionary traits, especially for finding late-stage stellar populations (Chilingarian & Zolotukhin, 2011; Chang et al., 2012).

ii. Significance of Color Index Distributions

Subgroup identification and classification depend heavily on the distributions of the g-r and r-i color indices. The bimodal stellar populations found in the CMDs, such as main-sequence stars and giants, are well-aligned with the peak g-r distribution, which has peaks at 0.9 and 1.8 (Maraston et al., 2011; Belloni et al. By showing clear differences between subgroups, these distinct peaks make the clustering model more accurate, as seen in Section II.A of the CMD patterns (Fan et al., 2018). Also, the small range of r-i values centered around 0.4 shows that the dataset can effectively find cooler, late-type stellar populations. This finding fits with the r-i vs. i CMD (Figure 3), which shows how evolved stars are (Carretta et al., 2011).

The clear color index distributions make it possible for unsupervised learning tasks like clustering, improving silhouette scores and inter-cluster distances by making it easy to tell subgroups apart. The narrow r-i spread ensures reliable subgroup classification of cooler stars, while the bimodal g-r distribution helps distinguish main-sequence stars from giants.

It is very important to have these indices in supervised classification tasks (Section IV). By connecting changes in color to chemical and kinematic signatures, the g-r and r-i indices help tell the difference between in situ and accreted populations (Maraston et al., 2011; Carretta et al., 2011). Their even distributions show very little observational bias, which helps classification models do well in terms of precision and recall (Li & Han, 2007; Belloni et al., 2017). The analysis of the Gaussian Mixture Model (GMM) and spread metrics for g-r and r-i adds to the significance of the Color Index Distributions section's depth. The GMM analysis of the g-r color index shows two modes with means of 1.22 and 0.43 and weights of 0.69 and 0.31. These results confirm the presence of main-sequence stars and giants, as well as the bimodality of the distribution of stellar subgroups (Reynolds, 2009; Tiwari et al., 2020). The primary mode's higher weight emphasizes the dataset's strong representation of common stellar populations, while the secondary mode captures less frequent but significant subgroups (Kawabata, 2008). To distinguish cooler, late-type stars, the interquartile range (IQR) of 0.65 for r-i emphasizes its narrow spread. The dataset's low coefficient of variation (CV) of 1.19 supports its clustering and classification robustness (Shen et al., 2018). These quantitative metrics improve the dataset's subgroup detection accuracy and the reliability of subsequent clustering models (Section III) and classification algorithms (Section IV) (Ay et al., 2021). These analyses strengthen the dataset's alignment with theoretical clustering and its ability to support advanced astrophysical studies from a quantitative perspective. Exploratory data analysis and complex clustering and classification methods are seamlessly linked, validating the utility of GMM in astrophysics and ensuring reliable subgroup detection across stellar populations (Tiwari et al., 2020).

D. Integration with Clustering and Classification:

The bimodal distribution in g-r (with cluster centers around 1.44 and 0.51) supports the separation of stellar groups, such as main-sequence stars and giants (Kuhn et al., 2015; Scrucca et al., 2016). Li et al. (2018) say that the peaks in the g-r histogram and the CMDs are very close to the centers of these clusters. A moderate silhouette score (0.65) that gauges how steady the cluster assignments are is produced by the clear separation between these centers, which supports the clustering models (Section III). In the classification tasks in Section IV, the found clusters are used as important training labels to help the model tell the difference between stellar subgroups (Ay et al., 2018). The weights (0.50 and 0.49), which show that the dataset matches known stellar population distributions, show that it can be used for both unsupervised and supervised learning methods (Kuhn et al., 2015). This study’s results serve as a bridge between EDA and advanced modeling, offering a quantitative foundation for clustering and classification methods. With these metrics, the dataset is aligned with both theoretical and practical stellar population models. This makes it even more useful for future astrophysical study.

E. Positional Data Analysis

i. Spatial Coverage

Examining the distribution of items in RA and Dec allows us to evaluate the dataset’s spatial coverage. The RA vs. Dec scatter plot (Figure 11) shows the distribution of stars and slight variations in star density across the spatial region (Gonidakis et al., 2008; Du et al., 2003).

a. Visual Insights

The scatter plot with overlay density (Figure 12) shows a mostly uniform distribution of stars, with higher density regions clustered close to the center. Peaks correspond to areas of slight clustering tendencies in the density gradient, which runs from 0.2 to 1.6. This suggests that there have been localized gains in stellar concentration, which may have something to do with galactic substructures. Figure 13: The density histogram shows that star densities are mostly spread out between 0.4 and 1.6, with 1.05 as the mean and 1.02 as the median. According to these findings, there are slight variations in density, but the overall distribution is stable (Santos et al., 2005). This is consistent with trends found in earlier research. Changes in density across spatial regions are measured by the grid-based density map (Figure 14). With the highest concentrations found in localized regions of RA and Dec., densities range from 0 to 30 stars per unit grid. Insights into the spatial properties of the dataset are provided by this visualization, which efficiently identifies both highly populated regions and areas with sparse coverage.

b. Quantitative Analysis

The dataset spans an RA range of 180.0005 to 180.9999 degrees and a Dec range of -0.4999 to 0.4201 degrees. The mean density is 1.13, and the median density is 1.15, with densities ranging between 0.21 to 1.59. The grid-based density values vary between 0 to 30 stars per grid cell, capturing local density nuances that confirm the dataset’s comprehensive sampling across the defined spatial region (Ruphy, 1998).

ii. Anomalies and Clustering

a. Visual Insights

The scatter map of RA vs. Dec with a cluster density overlay shows the general distribution of stars, highlighting areas of interest where clustering patterns appear (Gonidakis et al., 2008). Regions with high and low stellar densities are clearly marked on the Cluster-Highlighting Density Map. The top 5% of density numbers are represented by high-density regions marked in red, while the bottom 5% are represented by low-density regions marked in blue. According to the study, the RA range of 180.02 to 180.60 degrees and the Dec range of -0.35 to -0.10 degrees are where the majority of the high-density regions in the dataset are found. According to Izakian and Pedrycz (2014), these regions point to the possibility of localized stellar clusters or galactic substructures. On the other hand, low-density regions are spread out randomly along the edges of the spatial coverage, which is in line with what we would expect from regions with fewer people. We can find and understand localized clustering behaviors better with the new image. It gives us useful information about the dataset’s spatial structure and stellar distribution patterns.

b. Quantitative Analysis

The table below highlights the spatial characteristics and density metrics for high-density clusters derived from quantitative analysis. The dataset demonstrates reliability for clustering and positional analysis due to its consistent lack of significant spatial gaps (Krone-Martins & Moitinho, 2013).

The Cluster-Highlighting Density Map (Figure 15) provides a visual depiction of the high-density regions, showing their localized clustering within the dataset. High-density clusters, constituting 5.00% of the dataset, are primarily concentrated in the RA range of 180.16 to 180.80 degrees and the Dec range of -0.27 to -0.14 degrees. These regions likely represent stellar clusters or galactic substructures and highlight areas of interest for further clustering and classification tasks (Toloba et al., 2014).

III. Clustering

Clustering is a key method for categorizing stellar populations by shared traits in astronomical datasets (Kgoadi et al., 2017). This section uses K-Means and DBSCAN algorithms to analyze photometric g-r, r-i, and positional (RA, Dec) data to find localized structures and work with high-dimensional data (Zhang & Zhao, 2004). By examining clustering metrics like silhouette scores, inter-cluster distances, and cluster size distribution (Rebbapragada et al., 2009), we assess the accuracy and dependability of these methods in identifying stellar subgroups. The clustering results, which match theoretical stellar classifications, provide insights into Galactic substructures and photometric variability.

A. K-Means Clustering: g-r vs. r-i

Figure 16 shows K-Means clustering results for g-r versus r-i photometric space. The cluster showed four stellar clusters, each exhibiting distinct stellar property variations. The algorithm efficiently categorized stars into broad categories that presumably correlate to various stellar populations, such as main sequence giants and giants, despite considerable overlap across the clusters. The cluster centers are situated at (g-r = 1.48, r-i = 1.27), (g-r = 1.72, r-i = -0.63), (g-r = 0.49, r-i = 0.16), and (g-r = 1.29, r-i = 0.54), corresponding to distinct population subsets within the dataset. This clustering has a silhouette score of 0.477, suggesting modest performance. As a recurrent problem in the analysis of stellar populations with similar photometric features, the clusters overlap, even if they are distinct. Such results are consistent with other research, which emphasize the inherent uncertainty in cluster borders owing to stellar characteristics’ continuous nature (Kgoadi et al., 2017; Bazarghan, 2012).

Figure 16 shows the clustering results, with four groups identified by distinct colors. This star cluster visualization shows the structure of the cluster space and reveals the clustering patterns of the stars, enabling additional astrophysical property analysis.

B. DBSCAN Clustering: RA vs Dec

Figure 17 shows DBSCAN clustering results for RA and Dec positional data. We found no noise spots in DBSCAN’s single cluster analysis of the whole dataset. The stellar density distribution in the spatial area is very uniform, missing the distinct variations needed for DBSCAN to distinguish clusters or identify localized stellar groups. The lack of noise points and a single cluster indicate that the selected parameters (epsilon = 0.5, min samples = 10) may not capture relevant substructures in the dataset. The silhouette score needs more than one cluster to determine clustering quality; hence it is not relevant here. This matches the ocular observation of a uniformly distributed dataset, where DBSCAN cannot detect clustering patterns. While DBSCAN has been shown to be efficient in discovering galactic clusters in previous research, its limitations in this analysis emphasize the difficulties of applying density-based clustering algorithms to datasets with low density variations or uniform sampling (Izakian & Pedrycz, 2014; Kurban et al., 2017).

C. K-Means Clustering: RA vs. Dec

This scatter plot shows K-Means clustering of RA and Dec positional data. Four distinct clusters are found in the analysis, equally distributed throughout the spatial area. The cluster centers are placed at (180.75, -0.25), (180.24, 0.31), (180.25, -0.25), and (180.76, 0.31), indicating a clear quadrant split of the dataset. The silhouette score of 0.478 indicates intermediate clustering quality, with well-defined clusters overlapping at their boundaries. The dataset’s uniform sampling and low density variations make K-Means good for partitioning but not for localized clustering or finer substructures (Rebbapragada et al., 2009; Zhang & Zhao, 2004). The plot demonstrates how strictly the K-Means algorithm divides data since cluster boundaries are geometric rather than density-driven. This approach divides the dataset into regions; however, it may overlook density variations that indicate galactic substructures or other astrophysical events. These high-density results show the virtues and weaknesses of K-Means clustering for positional data, especially compared to density-based approaches like DBSCAN.

D. DBSCAN Clustering: g-r vs. r-i

The plot (Figure 19) shows DBSCAN clustering results in the photometric space (g-r and r-i) (Izakian & Pedrycz, 2014; Kurban et al., 2017). For two clusters, the algorithm labeled 103 points as noise (red). Cluster 0 contains most of the data, highlighting a dense collection of stars with similar photometric features. Cluster 1, having fewer points, is a smaller dataset substructure. The presence of noise points (2% of the dataset) makes differentiating clusters in photometric space where stellar populations overlap difficult (Tramacere et al., 2012; Sapozhnikov & Kovaleva, 2019). These noise points may be abnormal stars with unusual photometric attributes, uncommon stellar kinds, or measurement mistakes. Due to noise and a low number of valid clusters, the silhouette score cannot be used to measure clustering quality (Prisinzano et al., 2021). These clustering results show that DBSCAN may have trouble resolving photometric clusters for this dataset. Identifying distinct groups is difficult owing to the high density of overlapping populations in the g-r and r-i photometric space (Castro-Ginard et al., 2020). Localized substructures and regions of photometric variation are effectively identified by the approach.

IV. Classification

Classification is a crucial technique for classifying stellar populations by photometric and positional characteristics. Using photometric magnitudes, color indices, and star positional data, this section uses Random Forest and SVM algorithms to categorize stars into distinct stellar populations. We evaluate these approaches’ dependability and efficacy in discriminating in situ and accreted populations using classification metrics including accuracy, precision, recall, and F1-score.

A. Random Forest Classification

The Random Forest star classification algorithm was used to categorize giants, main sequence stars, and white dwarfs using photometric features such as g-r, u-g, and r-i (Costa-Duarte et al., 2019; Carliles et al., 2007). The model exhibits 99.9% classification accuracy and near-perfect metrics. Below is the classification report:

Based on population and recall metrics, the model successfully finds stellar populations. The model correctly recognized all giants without creating any false positives or false negatives, as shown by the model’s superb precision and recall scores (Wen et al., 2020). White dwarfs had classification and F1-scores close to 1.0, whereas main sequence stars got ratings of 0.999 with few misclassifications. These results show the model’s exceptional accuracy in identifying these stellar groups (Plewa, 2018). The feature importance plot (Figure 20) displays how each feature influences model decision-making. For features, the g-r color index accounts for 84% of overall importance. This underscores the importance of g-r in identifying stellar populations based on their photometric features (Yi & Pan, 2010). The importance of the two features, u-g 7.2% and r-i 7.6%, is substantially lower than g-r. Photometric features are more essential than positional data for classification tasks than RA 0.55% and Dec 0.52% (Costa-Duarte et al., 2019). The Random Forest classifier excels at utilizing stellar populations to categorize stellar populations with high precision and reliability. By capturing variations in stellar temperature and composition, the main predictor, g-r, agrees with astrophysical expectations (Carliles et al., 2007). A balanced classification metric and high accuracy make the algorithm suited for evaluating Galactic stellar populations. Our results demonstrate Random Forest’s reliability and interpretability for photometric datasets on Galactic substructures and stellar evolution (Plewa, 2018).

B. Support Vector Machine (SVM) Classification

Using a radial basis function (RBF) kernel, the SVM classification was applied to positional RA, Dec, and photometric data, yielding good performance metrics (Hastie, Tibshirani, & Friedman, 2009). Hyperparameter optimization yielded 98.8% classification accuracy with C = 100 and gamma = 0.1 as optimal parameters (Schölkopf & Smola, 2002). The classification report reveals precision, recall, and F1 scores around 0.99 for giants, main-sequence stars, and white dwarfs. These high population metrics show the model’s resilience in differentiating stellar populations (Wen et al., 2020). Figures 21 and 22 show confusion matrices, a visual depiction of the model’s predictions vs. true labels (Pedregosa et al., 2011). The matrix’s diagonal values (135 giants, 611 main-sequence stars, and 242 white dwarfs) are valid predictions, whereas off-diagonal values are misclassifications. In 1,000 test data, the model classified stars with just 10 misclassifications across all categories. Due to the SVM’s high recall of 0.98 for giants and white dwarfs and 0.99 for main-sequence stars, these results match the published metrics (Bishop, 2006). The charts show balanced performance across courses, with no stellar population bias. SVMs use photometric and positional data to capture complicated correlations, resulting in high accuracy (Cortes & Vapnik, 1995). The RBF kernel translates non-linear decision boundaries for exact classification in multidimensional feature space. These results show that SVM can accurately analyze photometric and positional information, revealing Galactic stellar populations (Vapnik, 1998).

The main difference between the two confusion matrix displays is how the colors are organized. Figure 21 uses a formal blue-to-light-blue gradient for low presentation, whereas Figure 22 uses a vibrant yellow-green-blue gradient for contrast. Design and readability vary, but both show classification data.

V. Validation Outcomes

A. CMD-Based Validation of Clusters

CMD-based validation of clusters reveals clustering results and theoretical cluster track alignment (Valle et al., 2021). Figure 23 shows the g-r color index vs. r magnitude distribution of stellar populations. Photometrically distinct clusters are labeled Cluster 0, Cluster 1, and Cluster 2. The alignment of the cluster isochrone in red with the main stellar sequence proves clustering’s accuracy in collecting stellar evolutionary histories (Bussola et al., 2012). Cluster 0 in the lower CMD has stars with lower magnitudes and higher g-r color indices, suggesting fainter, cooler stellar dwarfs or stars in advanced evolutionary stages (Koleva et al., 2008). The isochrone matches Cluster 1’s main-sequence stars’ modest magnitudes and color indices. emphasizing the robustness of the clustering method. Top Cluster 2 features brighter, hotter stellar populations, likely corresponding to early-type main sequence stars or younger stellar populations (Kaur & Joshi, 2022). High photometric readings and strong clustering are evidenced by low cluster overlap and the absence of noteworthy outliers (Li & Shao, 2021). The clusters’ alignment with the theoretical isochrone reveals that the clustering algorithm effectively identifies subgroups in the dataset. These results confirm the dataset’s suitability for studying Galactic stellar populations and highlight the clustering model’s utility in astrophysical assessments of stellar evolution and structure (Kerber et al., 2006).

B. Comparison with Known Galactic Substructures

Figure 22 shows the clustering results and a known Galactic cluster projected into Principal Component Analysis (PCA) space using positional data (Wang et al., 2024). The stellar substructures are shown in their reduced dimensions, with Galactic markers indicating the positions of the stellar population Gaia-Enceladus, the Sagittarius Stream, and the Thick Disk (Li et al., 2023). The scatter plots show the stellar population distribution and theoretical substructures (Helmi et al., 2018). Galactic substructures are clearly marked on the plot, providing a distinct visual of their locations in PCA space (Peñarrubia & Petersen, 2021). The Gaia-Enceladus substructure correlates with a denser stellar distribution at PCA1 = -0.50 and PCA2 = -0.20. At low stellar density, the Sagittarius Stream appears at PCA1 = 0.30 and PCA2 = 0.50 (Liu et al., 2024). As expected around the core stellar population density, the thick disk is centrally centered at PCA1 = 0.00 and PCA2 = 0.00. The dataset’s representation of known Galactic components and the clustering algorithm’s ability to split subpopulations are confirmed by these substructure alignments (Helmi et al., 2018). PCA-derived coordinates of Galactic substructures and their high distribution relative to stellar population are shown down below. The clustering approach converts these structures into the dataset’s PCA space, enabling quantitative analysis of Galactic substructures (Li et al., 2023).

5. Discussion

I. Data Preparation and Quality Analysis

The dataset preparation technique is dedicated to ensuring high-quality results; however, its strengths require examination (Polsterer et al., 2014). The limited RA and Dec ranges used for this analysis prevent contamination from unrelated stellar populations (Kremer et al., 2015). Increasing data coherence with this spatial restriction may lead to missing galactic features or patterns across regions (Sanjaripour et al., 2024). Astronomers balance precision and generalizability to results. The homogeneity distribution in the RA and Dec distributions is positive since it suggests that the specified spatial area is devoid of observational biases and artifacts, which is necessary for clustering and classification tasks (Montero et al., 2010). Researchers can examine stellar populations with photometric precision in the u, g, r, i, and z bands (McNeil & Moody, 2005). Median and range magnitudes show accurate stellar brightness for average and edge-case populations (Sanjaripour et al., 2024). However, photometric error outliers like the high u-band value need more study (Jia et al., 2024). Outliers may indicate observational or reduction method problems despite having little statistical influence. Error modeling or recalibration may improve photometric data reliability (Polsterer et al., 2014). Quality filters improve datasets but prejudice them (Kremer et al., 2015). To retain high-quality data, half the entries were removed, which may have omitted dim or uncommon astrophysically relevant stellar populations (Sanjaripour et al., 2024). The remaining 2,536 entries balance reliability and sample size; however, they may have reduced the dataset’s capacity to find distinct or poorly defined subpopulations (McNeil & Moody, 2005). The use of thresholds and flags, although beneficial, underscores the importance of openness and repeatability in the preprocessing pipeline to offer independent validation of results (Humphrey et al., 2022). Statistical observations like distributions and cluster coverage uniformity provide clustering and classification tasks to the dataset (Sanjaripour et al., 2024). These strengths are high on the necessity for thorough preparation in such an analysis (Polsterer et al., 2014). The consistency and high reliability of the dataset lend confidence in the results but caution in interpretation. Although the dataset’s single spatial location and strict quality requirements may improve precision, they may restrict its applicability in other galactic situations (Kremer et al., 2015). Comments show how data preparation influences scientific problems and analysis (McNeil & Moody, 2005).

II. EDA: Insights from CMDs and Color Indices

CMDs help identify stellar populations and validate dataset alignment with theoretical stellar evolutionary models. The g−r vs. r CMD efficiently separates main sequence stars from giant stars, revealing a continuous sequence indicative of stars in their hydrogen-burning phase. A well-defined main sequence and few outliers indicate the dataset’s high quality and theoretical alignment (Kalirai & Tosi, 2004; Nardiello et al., 2018). This exact stellar phase mapping identifies evolutionary tracks, proving the dataset’s applicability for clustering and classification investigations (Li & Han, 2007). Using the u−g vs. g CMD, the dataset may record early evolutionary periods in hotter, younger stellar populations (Chilingarian & Zolotukhin, 2011). Using theoretical models for ultraviolet-bright stars, the data shows stars spreading along evolutionary tracks (Baraffe et al., 1998). These results show the CMD’s ability to characterize high-energy stellar populations and the dataset’s diversity in evolutionary phases (Ferraro et al., 1996). The dataset’s alignment with theoretical models is further supported by the observed bimodal distributions in color indices like g−r and r−i. These distributions include stellar population groupings, such as main sequence stars and giants, highlighting the dataset’s ability to capture major stellar population variations (Milone et al., 2016; Bellini et al., 2009). A well-prepared dataset is needed for accurate subgroup identification and classification tasks, as shown by the equal dispersion within each peak and the lack of observational bias (Ree et al., 2001). Integration of CMDs and color index distributions improves the dataset’s alignment with theoretical stellar evolutionary tracks. These methods offer a solid clustering and classification framework by mapping evolutionary trajectories and differentiating subgroups (Gao et al., 2012). To minimize possible biases in understanding stellar population dynamics, the dependence on unambiguous theoretical alignments further underscores the importance of dataset precision. This talk stresses the crucial importance of CMDs and color indices in our knowledge of Galactic stellar populations via extensive and trustworthy analysis (Li et al., 2017).

III. Clustering Techniques and Galactic Substructure Identification

By clustering stars by photometric and positional attributes, K-Means and DBSCAN study stellar populations. In order to depict stellar groupings, the K-Means cluster separates the dataset into distinct clusters. In the g-r vs. r-i space, photometric data clustering gave four stellar scores with silhouette values showing significant separation. This score shows the algorithm’s ability to build cluster boundaries despite overlaps, a significant issue in continuous stellar property datasets (Kgoadi et al., 2017; Rebbapragada et al., 2009). K-Means may ignore minor galactic substructures by geometrically partitioning larger populations, decreasing its sensitivity to local density variations (Zhang & Zhao, 2004). However, DBSCAN shows density-based clustering patterns that suggest separate Galactic substructures. The uniform positional data dataset (RA vs. Dec) had just one cluster, highlighting DBSCAN’s inability to find significant patterns in uniformly distributed datasets (Izakian & Pedrycz, 2014). In photometric g-r vs. r-i space, DBSCAN found two clusters with low-noise points, demonstrating sensitivity to outliers. DBSCAN’s efficiency depends on parameter change and dataset features due to stellar populations’ high photometric overlap (Tramacere et al., 2012; Sapozhnikov & Kovaleva, 2019). Clustering metrics, especially score metrics, show the reliability of clustering approaches. K-Means clustering in positional data with a silhouette quality score of 0.478 implies moderate clustering with considerable boundaries overlap. This intermediate score underscores the method’s usefulness in distinguishing datasets with consistent density distributions but raises questions about its use with datasets with more complex density distributions (Zhang & Zhao, 2004). In single-cluster outputs, DBSCAN’s success is not measured by silhouette scores, highlighting the difficulty of applying density-based algorithms to datasets with low density variations (Castro-Ginard et al., 2020). These clustering methods show the complexity of identifying Galactic substructure clusters. K-Means uses centroid-based boundaries to generate global stellar population cluster categories inefficiently, which may mask density-driven phenomena like tidal streams or limited clusters (Kremers et al., 2021). DBSCAN recognizes such features better but needs parameter calibration and is limited to uniform density datasets (Bajal et al., 2021). These results show that a hybrid algorithm or advanced clustering techniques are needed to uncover Galactic substructures and fit theoretical models and empirical assumptions.

IV. Classification of Stellar Populations

Random Forest and SVM stellar population classification classify stars as main sequence, giants, or white dwarfs. With a 99.9% classification accuracy score, the Random Forest model can identify these populations with near-perfect population, recall, and F1-recall. Classifying giants, main sequence stars, and white dwarfs with low misclassifications proves its endurance. The g-r color index, which accounts for 84% of photometric classification, underscores the importance of feature features. This supports astrophysical notions that stellar populations vary in temperature and composition owing to population variations (Carliles et al., 2007; Costa-Duarte et al., 2019; Plewa, 2018). The machine learning SVM model with the RBF kernel performs with 98.8% accuracy. Its precision, recall, and F1 scores for giants, main sequence stars, and white dwarfs show balanced classification at all levels. The model’s robustness and fine-tuned decision boundaries in multidimensional feature space result in low misclassification rates of 10 in 1,000 test samples. SVM hyperparameter change, such as C = 100, helps it handle non-linear separations and find complex dataset patterns (Schölkopf & Smola, 2002; Vapnik, 1998; Yi & Pan, 2010). The feature results show a high dependence on positional characteristics. over photometric data. Since photometric magnitudes and color indices directly measure stellar temperature, composition, and evolutionary phases, they are optimal for the dataset. Decreased importance in classification tasks shows that positional features are better for clustering and cluster analysis than categorical assignments (Yi & Pan, 2010; Plewa, 2018). Each model has merits and downsides. Despite its interpretability and reliability, Random Forest’s feature dominance may hide subtler correlations. SVM calibration decreases overfitting and misclassifications due to kernel settings. These feature models show the importance of integrating feature-based and boundary-based classification approaches in order to appropriately identify galactic stellar populations. The dataset’s results show alignment with theoretical models and advanced machine learning technologies’ astrophysical discoveries (Costa-Duarte et al., 2019; Plewa, 2018; Carliles et al., 2007).

V. Validation of Clusters

Clustering results’ endurance and astrophysical importance are confirmed by CMD clusters and comparisons with Galactic substructures. The alignment of photometrically produced clusters with theoretical stellar evolutionary tracks in Figure 23 provides a strong background for evaluating clustering methodologies (Valle et al., 2021; Bussola et al., 2012). The distinct separation of Cluster 0, Cluster 1, and Cluster 2 in the g-r color index vs. r magnitude enhances the algorithm’s ability to discern between stellar populations at different evolutionary stages. CMD Cluster 0, with fainter, cooler dwarfs, maps late-stage stellar evolution models, whereas Cluster 1 maps main stellar sequences. Cluster 2 shows how the approach may uncover younger stellar populations with brighter, hotter stars (Koleva et al., 2008; Kaur & Joshi, 2022). Minimum overlap between clusters and no major cluster imply dataset quality and clustering accuracy (Li & Shao, 2021). Figure 24 shows a galactic computer analysis of clustering results vs. substructures. PCA space substructures like Gaia-Enceladus, the Sagittarius Stream, and the Thick Disk match stellar populations (Wang et al., 2024). Gaia-Enceladus is affected by strong clustering at PCA1 = -0.50 and PCA2 = -0.20. The Sagittarius Stream’s placement at PCA1 = 0.30 and PCA2 = 0.50 shows the dataset’s capacity to follow low-density tidal features, while the Thick Disk’s center position shows its dominance among local Galactic components (Helmi et al., 2018; Liu et al., 2024). An alignment of PCA-derived clusters with Galactic substructures shows that the clustering model can identify hierarchical cluster production from complicated stellar dynamics (Peñarrubia & Petersen, 2021; Li et al., 2023). The results influence the Galactic evolution theory. CMD and PCA analysis suggest the dataset may find evolutionary stellar tracks and large-scale substructures (Helmi et al., 2018; Valle et al., 2021). This twofold confirmation strengthens the clustering method’s suitability for astrophysical studies like Milky Way chemical and dynamical history. The research uses stellar evolution modeling and large-scale Galactic archaeology to link photometric and positional clustering results to theoretical frameworks and observable Galactic substructures (Kaur & Joshi, 2022; Peñarrubia & Petersen, 2021). The clustering model’s performance underscores its promise as a trustworthy instrument for studying the interplay of star formation, migration, and Galactic structure assembly. The dataset’s precision in finding features like Gaia-Enceladus and the Sagittarius Stream is noteworthy for revealing weak, low-density substructures that larger surveys miss (Li et al., 2023). This enables investigations on stellar populations, Milky Way accretion, and dynamical evolution. To analyze stellar populations and Galactic morphology, CMD alignment and substructure projection confirmation are required. These studies show how advanced clustering approaches uncover Galactic evolution’s complexity (Liu et al., 2024; Helmi et al., 2018).

6. Learning and Follow-Up

Our research shows that stellar populations can be estimated using clean photometric data and a modular system. Using stellar clustering and advanced sequence classification techniques, main sequence stars, giants, and white dwarfs were found. The CMD and PCA-based validations aligned cluster substructures with theoretical stellar evolutionary tracks and known cluster substructures, demonstrating the method’s effectiveness (Nardiello et al., 2014; Brasseur et al., 2010). The astrophysical study stressed the ability of photometric and positional data to reveal complex stellar movements and structures (Brown et al., 2008; Popescu & Hanson, 2008). Problems with data preparation and clustering slowed down research. Tight population filters ensured a high reliability of stellar populations while ensuring a low dataset size, possibly missing rare or faint stellar populations that could offer important insights (Hartmann et al., 2022). For techniques like DBSCAN’s parameter resizability and K-Means’ geometric stiffness to work, they need a mixed or flexible algorithm. These concerns might improve with the discovery of small substructures and stellar dynamics (Zoccali et al., 2002; Plewa, 2018). Moreover, to improve generalizability, our dataset could benefit from the addition of sky regions. This would make it possible to study galactic structures and substructures in greater detail outside of this spatial region (Nardiello et al., 2014). In particular, for complex or overlapped stellar populations, advanced machine learning techniques, such as neural networks or ensemble clusters, may improve stellar classification and clustering accuracy (Hartmann et al., 2022; Brasseur et al., 2010). It’s possible to improve models by changing hyperparameters and adding astrophysical constraints (Plewa, 2018). Putting together spectroscopic and photometric data is an interesting area of research. Using stellar metallicity, motion, and chemical data, the beginnings and evolution of the Milky Way may be better known (Zoccali et al., 2002; Popescu & Hanson, 2008). Photometric clustering and classification would be included in a full Galactic forensics system. The system might be able to handle very large datasets with the help of real-time data merging processes from future studies like the Vera Rubin Observatory, ensuring its use in big data in astronomy (Brown et al., 2008; Plewa, 2018).

References

-, S. S., -, Priyadharshini. P., & -, Adhilakshmi. M. (2024). Comparative Evaluation of K-Means, Hierarchical Clustering, and DBSCAN in Blood Donor Segmentation. International Journal For Multidisciplinary Research, 6(4), 26755. https://doi.org/10.36948/ijfmr.2024.v06i04.26755

1.4. Support Vector Machines. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/svm.html

6.3. Preprocessing data — Scikit-learn 1.5.2 documentation. (n.d.). Retrieved December 4, 2024, from https://scikit-learn.org/1.5/modules/preprocessing.html

[1210.0522] gamma-ray DBSCAN: a clustering algorithm applied to Fermi-LAT gamma-ray data. I. Detection performances with real and simulated data. (n.d.). Retrieved December 4, 2024, from https://arxiv.org/abs/1210.0522

accuracy_score in Sklearn — Javatpoint. (n.d.). Retrieved December 4, 2024, from https://www.javatpoint.com/accuracy_score-in-sklearn

Acharya, V., Bora, P. S., Navin, K., Nazareth, A., Anusha, P. S., & Rao, S. (2018). Classification of SDSS Photometric Data Using Machine Learning on A Cloud. Current Science, 115(2), 249. https://doi.org/10.18520/cs/v115/i2/249-257

An, D., Beers, T. C., Johnson, J. A., Pinsonneault, M. H., Lee, Y. S., Bovy, J., Ivezić, Ž., Carollo, D., & Newby, M. (2013). THE S℡LAR METALLICITY DISTRIBUTION FUNCTION OF THE GALACTIC HALO FROM SDSS PHOTOMETRY. The Astrophysical Journal, 763(1), 65. https://doi.org/10.1088/0004-637X/763/1/65

Annis, J., Soares-Santos, M., Strauss, M. A., Becker, A. C., Dodelson, S., Fan, X., Gunn, J. E., Hao, J., Ivezić, Ž., Jester, S., Jiang, L., Johnston, D. E., Kubo, J. M., Lampeitl, H., Lin, H., Lupton, R. H., Miknaitis, G., Seo, H.-J., Simet, M., & Yanny, B. (2014). THE SLOAN DIGITAL SKY SURVEY COADD: 275 deg2 OF DEEP SLOAN DIGITAL SKY SURVEY IMAGING ON STRIPE 82. The Astrophysical Journal, 794(2), 120. https://doi.org/10.1088/0004-637X/794/2/120

Anomaly Detection and Characterization in Spatial Time Series Data: A Cluster-Centric Approach | IEEE Journals & Magazine | IEEE Xplore. (n.d.). Retrieved December 1, 2024, from https://ieeexplore.ieee.org/document/6722892

Application of random forest to stellar spectral classification | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/5648041

Application of self-organizing map to stellar spectral classifications | Astrophysics and Space Science. (n.d.). Retrieved December 1, 2024, from https://link.springer.com/article/10.1007/s10509-011-0822-7

Bajal, E., Katara, V., Bhatia, M., & Hooda, M. (n.d.). A Review of Clustering Algorithms: Comparison of DBSCAN and K-mean with Oversampling and t-SNE. Recent Patents on Engineering, 16(2), 17–31. https://doi.org/10.2174/1872212115666210208222231

Baraffe, I., Chabrier, G., Allard, F., & Hauschildt, P. (1998). Evolutionary models for solar metallicity low — mass stars: Mass — magnitude relationships and color — magnitude diagrams. Astronomy and Astrophysics. https://www.semanticscholar.org/paper/Evolutionary-models-for-solar-metallicity-low-mass-Baraffe-Chabrier/4a3ce8c701ea67ab2f7a3fddd180a41fa8713fa3?utm_source=consensus

Beccari, G., Petr-Gotzens, M. G., Boffin, H. M. J., Romaniello, M., Fedele, D., Carraro, G., De Marchi, G., De Wit, W.-J., Drew, J. E., Kalari, V. M., Manara, C. F., Martin, E. L., Mieske, S., Panagia, N., Testi, L., Vink, J. S., Walsh, J. R., & Wright, N. J. (2017). A tale of three cities: OmegaCAM discovers multiple sequences in the color-magnitude diagram of the Orion Nebula Cluster. Astronomy & Astrophysics, 604, A22. https://doi.org/10.1051/0004-6361/201730432

Beck, R., Dobos, L., Budavári, T., Szalay, A. S., & Csabai, I. (2016). Photometric redshifts for the SDSS Data Release 12. Monthly Notices of the Royal Astronomical Society, 460(2), 1371–1381. https://doi.org/10.1093/mnras/stw1009

Belloni, D., Askar, A., Giersz, M., Kroupa, P., & Rocha-Pinto, H. J. (2017). On the initial binary population for star cluster simulations (No. arXiv:1707.04271). arXiv. https://doi.org/10.48550/arXiv.1707.04271

Best Python libraries for Machine Learning. (2019, January 18). GeeksforGeeks. https://www.geeksforgeeks.org/best-python-libraries-for-machine-learning/

Borsato, N. W., Martell, S. L., & Simpson, J. D. (2020). Identifying stellar streams in Gaia DR2 with data mining techniques. Monthly Notices of the Royal Astronomical Society, 492(1), 1370–1384. https://doi.org/10.1093/mnras/stz3479

Bovy, J., Rix, H.-W., Liu, C., Hogg, D. W., Beers, T. C., & Lee, Y. S. (2012). THE SPATIAL STRUCTURE OF MONO-ABUNDANCE SUB-POPULATIONS OF THE MILKY WAY DISK. The Astrophysical Journal, 753(2), 148. https://doi.org/10.1088/0004-637X/753/2/148

Bramich, D. M., & Freudling, W. (2012). Systematic trends in Sloan Digital Sky Survey photometric data: Systematic trends in SDSS photometric data. Monthly Notices of the Royal Astronomical Society, 424(2), 1584–1599. https://doi.org/10.1111/j.1365-2966.2012.21385.x

Bramich, D. M., Horne, K., Albrow, M. D., Tsapras, Y., Snodgrass, C., Street, R. A., Hundertmark, M., Kains, N., Arellano, F. A., Figuera, J. R., & Giridhar, S. (2013). Difference image analysis: Extension to a spatially varying photometric scale factor and other considerations. Monthly Notices of the Royal Astronomical Society, 428(3), 2275–2289. https://doi.org/10.1093/mnras/sts184

Bussola, F., Faccioli, M., Frezzato, G., & Romanelli, G. (2012). Photometric determination of the age and distance of the open cluster NGC 2420. https://www.semanticscholar.org/paper/Photometric-determination-of-the-age-and-distance-Bussola-Faccioli/922f16aced9000167c0926551398580e6b074f89?utm_source=consensus

Cabrera-Lavers, A., Garzón, F., & Hammersley, P. (2004). Stellar Distribution in the Galactic Disk from NIR Color-Magnitude Diagrams. In E. J. Alfaro, E. Pérez, & J. Franco (Eds.), How does the Galaxy Work? (pp. 221–224). Springer Netherlands. https://doi.org/10.1007/1-4020-2620-X_44

Campos, F., Kepler, S. O., Bonatto, C., & Ducati, J. R. (2013). Multichromatic colour-magnitude diagrams of the globular cluster NGC 6366. Monthly Notices of the Royal Astronomical Society, 433, 243–250. https://doi.org/10.1093/mnras/stt719

Cantat-Gaudin, T., Jordi, C., Vallenari, A., Bragaglia, A., Balaguer-Núñez, L., Soubiran, C., Bossini, D., Moitinho, A., Castro-Ginard, A., Krone-Martins, A., Casamiquela, L., Sordo, R., & Carrera, R. (2018). A Gaia DR2 view of the Open Cluster population in the Milky Way (No. arXiv:1805.08726). arXiv. https://doi.org/10.48550/arXiv.1805.08726

Carliles, S., Budav’ari, T., Heinis, S., Priebe, C., & Szalay, A. (2007). Photometric Redshift Estimation on SDSS Data Using Random Forests. arXiv: Astrophysics. https://www.semanticscholar.org/paper/Photometric-Redshift-Estimation-on-SDSS-Data-Using-Carliles-Budav'ari/badc2bd8c17449018ad07e1d0816cd494fa47180?utm_source=consensus

Case Study | Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies. (n.d.). Retrieved December 1, 2024, from https://dl.acm.org/doi/10.1145/3148055.3149208

Castro-Ginard, A., Jordi, C., Luri, X., Álvarez Cid-Fuentes, J., Casamiquela, L., Anders, F., Cantat-Gaudin, T., Monguió, M., Balaguer-Núñez, L., Solà, S., & Badia, R. M. (2020). Hunting for open clusters in Gaia DR2: 582 new open clusters in the Galactic disc. Astronomy & Astrophysics, 635, A45. https://doi.org/10.1051/0004-6361/201937386

Chen, B., Liu, X., Yuan, H., Robin, A. C., Huang, Y., Xiang, M., Wang, C., Ren, J., Tian, Z., & Zhang, H. (2016). Constraining the Galactic structure parameters with the XSTPS-GAC and SDSS photometric surveys (No. arXiv:1609.08838). arXiv. https://doi.org/10.48550/arXiv.1609.08838

Chilingarian, I., & Zolotukhin, I. (2011). A universal ultraviolet-optical colour-colour-magnitude relation of galaxies (No. arXiv:1102.1159). arXiv. https://doi.org/10.48550/arXiv.1102.1159

Classification of SDSS objects. (n.d.). Retrieved December 4, 2024, from https://kaggle.com/code/manu123/classification-of-sdss-objects

Classification_report. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.metrics.classification_report.html

Clustering Analysis in the Wireless Propagation Channel with a Variational Gaussian Mixture Model | IEEE Journals & Magazine | IEEE Xplore. (n.d.). Retrieved November 30, 2024, from https://ieeexplore.ieee.org/document/8365753

Clustering stellar pairs to detect extended stellar structures | Proceedings of the International Astronomical Union | Cambridge Core. (n.d.). Retrieved December 4, 2024, from https://www.cambridge.org/core/journals/proceedings-of-the-international-astronomical-union/article/clustering-stellar-pairs-to-detect-extended-stellar-structures/155B1B15AD886AFF1339A92C84759BE8

Colour-Magnitude Diagrams of Galactic and Globular Clusters. Stellar Evolution and Abundances of the Elements | SpringerLink. (n.d.). Retrieved November 29, 2024, from https://link.springer.com/chapter/10.1007/978-1-4684-7598-2_26

Comparative Assessment of Pulsar Families using GMM and DPGMM | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved November 30, 2024, from https://ieeexplore.ieee.org/document/8907001

Compute Classification Report and Confusion Matrix in Python — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/compute-classification-report-and-confusion-matrix-in-python/

ConfusionMatrixDisplay. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html

Conroy, C., & Dokkum, P. G. van. (2016). PIXEL COLOR MAGNITUDE DIAGRAMS FOR SEMI-RESOLVED S℡LAR POPULATIONS: THE STAR FORMATION HISTORY OF REGIONS WITHIN THE DISK AND BULGE OF M31. The Astrophysical Journal, 827(1), 9. https://doi.org/10.3847/0004-637X/827/1/9

Consolandi, G., Gavazzi, G., Fumagalli, M., Dotti, M., & Fossati, M. (2016). Robust automatic photometry of local galaxies from SDSS: Dissecting the color magnitude relation with color profiles⋆. Astronomy & Astrophysics, 591, A38. https://doi.org/10.1051/0004-6361/201527618

Costa-Duarte, M., Sampedro, L., Molino, A., Xavier, H. S., Herpich, F., Chies-Santos, A., Barbosa, C. E., Cortesi, A., Schoenell, W., Kanaan, A., Ribeiro, T., Oliveira, C., Akras, S., Alvarez-Candal, A., Barbosa, C., Castell’on, J. L. N., Coelho, P., Dantas, M., Dupke, R., … Souza, R. C. T. (2019). The S-PLUS: A star/galaxy classification based on a Machine Learning approach. arXiv: Astrophysics of Galaxies. https://www.semanticscholar.org/paper/The-S-PLUS%3A-a-star-galaxy-classification-based-on-a-Costa-Duarte-Sampedro/8d188b23ca139387ae7b528ab429733207d944da?utm_source=consensus

Das, A. K., Pati, S. K., & Ghosh, A. (2020). Relevant feature selection and ensemble classifier design using bi-objective genetic algorithm. Knowledge and Information Systems, 62(2), 423–455. https://doi.org/10.1007/s10115-019-01341-6

Data Access for SDSS DR17 Overview | SDSS. (n.d.). Retrieved December 4, 2024, from https://www.sdss4.org/dr17/data_access/

Data Mining and Machine Learning in Astronomy — Astrophysics Data System. (n.d.). Retrieved November 27, 2024, from https://ui.adsabs.harvard.edu/abs/2010IJMPD..19.1049B/abstract

Data Pre-Processing with Sklearn using Standard and Minmax scaler — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/data-pre-processing-wit-sklearn-using-standard-and-minmax-scaler/

DBSCAN. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.cluster.DBSCAN.html

Deep point spread function photometric catalog of the VVV survey data | Astronomy & Astrophysics (A&A). (n.d.). Retrieved November 27, 2024, from https://www.aanda.org/articles/aa/full_html/2019/12/aa35513-19/aa35513-19.html

Demidova, L. A., Klyueva, I. A., & Pylkin, A. N. (2019). Hybrid Approach to Improving the Results of the SVM Classification Using the Random Forest Algorithm. Procedia Computer Science, 150, 455–461. https://doi.org/10.1016/j.procs.2019.02.077

Desai, S., Mohr, J. J., Bertin, E., Kümmel, M., & Wetzstein, M. (2016). Detection and removal of artifacts in astronomical images. Astronomy and Computing, 16, 67–78. https://doi.org/10.1016/j.ascom.2016.04.002

Ding, C., & He, X. (2004). K-means clustering via principal component analysis. Proceedings of the Twenty-First International Conference on Machine Learning, 29. https://doi.org/10.1145/1015330.1015408

Domínguez Sánchez, H., Margalef, B., Bernardi, M., & Huertas-Company, M. (2021). SDSS-IV DR17: Final release of MaNGA PyMorph photometric and deep-learning morphological catalogues. Monthly Notices of the Royal Astronomical Society, 509(3), 4024–4036. https://doi.org/10.1093/mnras/stab3089

Dramiński, M., Rada-Iglesias, A., Enroth, S., Wadelius, C., Koronacki, J., & Komorowski, J. (2008). Monte Carlo feature selection for supervised classification. Bioinformatics, 24(1), 110–117. https://doi.org/10.1093/bioinformatics/btm486

Du, C., Zhou, X., Ma, J., Chen, A. B.-C., Yang, Y., Li, J., Wu, H., Jiang, Z., & Chen, J. (2003). Galactic structure studies with BATC star counts. Astronomy & Astrophysics, 407(2), 541–549. https://doi.org/10.1051/0004-6361:20030532

Efficient classification of portscan attacks using Support Vector Machine | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/6533915

Elorrieta, F., Eyheramendy, S., Jordán, A., Dékány, I., Catelan, M., Angeloni, R., Alonso-García, J., Contreras-Ramos, R., Gran, F., Hajdu, G., Espinoza, N., Saito, R. K., & Minniti, D. (2016). A machine learned classifier for RR Lyrae in the VVV survey. Astronomy & Astrophysics, 595, A82. https://doi.org/10.1051/0004-6361/201628700

Evans, D. W., Riello, M., Angeli, F. D., Carrasco, J. M., Montegriffo, P., Fabricius, C., Jordi, C., Palaversa, L., Diener, C., Busso, G., Cacciari, C., Leeuwen, F. van, Burgess, P. W., Davidson, M., Harrison, D. L., Hodgkin, S. T., Pancino, E., Richards, P. J., Altavilla, G., … Wyrzykowski, Ł. (2018). Gaia Data Release 2 — Photometric content and validation. Astronomy & Astrophysics, 616, A4. https://doi.org/10.1051/0004-6361/201832756

Experimental Analysis of Stellar Classification by using Different Machine Learning Algorithms | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/9952964

F8061038620 — International Journal of Recent Technology and Engineering (IJRTE). (n.d.). Retrieved December 4, 2024, from https://www.ijrte.org/portfolio-item/F8061038620/

Fan, Z., & Xu, X. (2019). Application and visualization of typical clustering algorithms in seismic data analysis. Procedia Computer Science, 151, 171–178. https://doi.org/10.1016/j.procs.2019.04.026

Ferraro, F. R., Carretta, E., Corsi, C. E., Pecci, F. F., Cacciari, C., Buonanno, R., Paltrinieri, B., & Hamilton, D. (1996). The stellar population of the Globular Cluster M 3. II: CCD photometry of additional 10,000 stars (No. arXiv:astro-ph/9611016). arXiv. https://doi.org/10.48550/arXiv.astro-ph/9611016

Feuillet, D. K., Feltzing, S., Sahlholdt, C., & Casagrande, L. (2020). The SkyMapper-Gaia RVS view of the Gaia-Enceladus-Sausage — An investigation of the metallicity and mass of the Milky Way’s last major merger (No. arXiv:2003.11039). arXiv. https://doi.org/10.48550/arXiv.2003.11039

fmarthoz. (2021, July 14). K-Means: Choosing the right number of clusters. Nerd For Tech. https://medium.com/nerd-for-tech/k-means-algorithm-in-4-parts-7540d0f33339

Fukugita, M., Ichikawa, T., Gunn, J. E., Doi, M., Shimasaku, K., & Schneider, D. P. (1996). The Sloan Digital Sky Survey Photometric System. The Astronomical Journal, 111, 1748. https://doi.org/10.1086/117915

Gado, N. E. I., Grall-Maës, E., & Kharouf, M. (2015). Linear KernelPCA and K-Means Clustering Using New Estimated Eigenvectors of the Sample Covariance Matrix. 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), 386–389. https://doi.org/10.1109/ICMLA.2015.207

Gaia Collaboration, Smart, R. L., Sarro, L. M., Rybizki, J., Reylé, C., Robin, A. C., Hambly, N. C., Abbas, U., Barstow, M. A., De Bruijne, J. H. J., Bucciarelli, B., Carrasco, J. M., Cooper, W. J., Hodgkin, S. T., Masana, E., Michalik, D., Sahlmann, J., Sozzetti, A., Brown, A. G. A., … Zwitter, T. (2021). Gaia Early Data Release 3: The Gaia Catalogue of Nearby Stars. Astronomy & Astrophysics, 649, A6. https://doi.org/10.1051/0004-6361/202039498

Gallart, C., Bernard, E. J., Brook, C. B., Ruiz-Lara, T., Cassisi, S., Hill, V., & Monelli, M. (2019). Uncovering the birth of the Milky Way through accurate stellar ages with Gaia. Nature Astronomy, 3(10), 932–939. https://doi.org/10.1038/s41550-019-0829-5

Gao, S., Just, A., & Grebel, E. K. (2013). Detailed comparison of Milky Way models based on stellar population synthesis and SDSS star counts at the north Galactic pole. Astronomy & Astrophysics, 549, A20. https://doi.org/10.1051/0004-6361/201118243

Gaussian Mixture Models (GMM) Covariances in Scikit Learn — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/gaussian-mixture-models-gmm-covariances-in-scikit-learn/

GaussianMixture. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.mixture.GaussianMixture.html

Generate classification report and confusion matrix in Python — ProjectPro. (n.d.). Retrieved December 4, 2024, from https://www.projectpro.io/recipes/generate-classification-report-and-confusion-matrix-in-python

Goessl, C. A., & Riffeser, A. (2001). Image reduction pipeline for the detection of variable sources in highly crowded fields (No. arXiv:astro-ph/0110704). arXiv. https://doi.org/10.48550/arXiv.astro-ph/0110704

Gonidakis, I., Livanou, E., Kontizas, E., Klein, U., Kontizas, M., Belcheva, M., Tsalmantza, P., & Karampelas, A. (2009). Structure of the SMC — Stellar component distribution from 2MASS data. Astronomy & Astrophysics, 496(2), Article 2. https://doi.org/10.1051/0004-6361/200809828

Goodness-of-fit test for isochrone fitting in the Gaia era — Statistical assessment of the error distribution | Astronomy & Astrophysics (A&A). (n.d.). Retrieved December 4, 2024, from https://www.aanda.org/articles/aa/full_html/2021/05/aa40413-21/aa40413-21.html

Heart Failure Prediction Using Artificial Intelligence Methods | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/10440664

Helmi, A., Babusiaux, C., Koppelman, H. H., Massari, D., Veljanoski, J., & Brown, A. G. A. (2018). The merger that led to the formation of the Milky Way’s inner stellar halo and thick disk. Nature, 563(7729), 85–88. https://doi.org/10.1038/s41586-018-0625-x

How to Interpret the Classification Report in sklearn (With Example). (n.d.). Retrieved December 4, 2024, from https://www.statology.org/sklearn-classification-report/

How to split the Dataset With scikit-learn’s train_test_split() Function — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/how-to-split-the-dataset-with-scikit-learns-train_test_split-function/

Hu, L., Liu, H., Zhang, J., & Liu, A. (2021). KR-DBSCAN: A density-based clustering algorithm based on reverse nearest neighbor and influence space. Expert Systems with Applications, 186, 115763. https://doi.org/10.1016/j.eswa.2021.115763

Huang, Z. (1998). Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values. Data Mining and Knowledge Discovery, 2(3), 283–304. https://doi.org/10.1023/A:1009769707641

Humphrey, A., Kuberski, W., Bialek, J., Perrakis, N., Cools, W., Nuyttens, N., Elakhrass, H., & Cunha, P. A. C. (2022). Machine-learning classification of astronomical sources: Estimating F1-score in the absence of ground truth. Monthly Notices of the Royal Astronomical Society: Letters, 517(1), L116–L120. https://doi.org/10.1093/mnrasl/slac120

Implementing DBSCAN algorithm using Sklearn. (2019, June 6). GeeksforGeeks. https://www.geeksforgeeks.org/implementing-dbscan-algorithm-using-sklearn/

Implementing PCA in Python with scikit-learn — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/implementing-pca-in-python-with-scikit-learn/

Introduction to k-Means Clustering with scikit-learn in Python | DataCamp. (n.d.). Retrieved December 4, 2024, from https://www.datacamp.com/tutorial/k-means-clustering-python

Ivezić, Ž., Lupton, R. H., Schlegel, D., Boroski, B., Adelman-McCarthy, J., Yanny, B., Kent, S., Stoughton, C., Finkbeiner, D., Padmanabhan, N., Rockosi, C. M., Gunn, J. E., Knapp, G. R., Strauss, M. A., Richards, G. T., Eisenstein, D., Nicinski, T., Kleinman, S. J., Krzesinski, J., … Lee, B. C. (2004). SDSS data management and photometric quality assessment. Astronomische Nachrichten, 325(6–8), 583–589. https://doi.org/10.1002/asna.200410285

Izakian, H., & Pedrycz, W. (2014). Anomaly Detection and Characterization in Spatial Time Series Data: A Cluster-Centric Approach. IEEE Transactions on Fuzzy Systems, 22(6), 1612–1624. IEEE Transactions on Fuzzy Systems. https://doi.org/10.1109/TFUZZ.2014.2302456

Jaffe, Y. L., Aragon-Salamanca, A., Lucia, G. D., Jablonka, P., Rudnick, G., Saglia, R., & Zaritsky, D. (2010). The colour-magnitude relation of Elliptical and Lenticular galaxies in the ESO Distant Cluster Survey (No. arXiv:1007.1425). arXiv. https://doi.org/10.48550/arXiv.1007.1425

Jin, X., & Hirakawa, K. (2012). Analysis and processing of pixel binning for color image sensor. EURASIP Journal on Advances in Signal Processing, 2012(1), 125. https://doi.org/10.1186/1687-6180-2012-125

Juric, M., Ivezic, Z., Brooks, A., Lupton, R. H., Schlegel, D., Finkbeiner, D., Padmanabhan, N., Bond, N., Sesar, B., Rockosi, C. M., Knapp, G. R., Gunn, J. E., Sumi, T., Schneider, D., Barentine, J. C., Brewington, H. J., Brinkmann, J., Fukugita, M., Harvanek, M., … York, D. G. (2008). The Milky Way Tomography with SDSS: I. Stellar Number Density Distribution (No. arXiv:astro-ph/0510520). arXiv. https://doi.org/10.48550/arXiv.astro-ph/0510520

Kalirai, J. S., & Tosi, M. (2004). Interpreting the Colour-Magnitude Diagrams of Open Star Clusters through Numerical Simulations (No. arXiv:astro-ph/0403420). arXiv. https://doi.org/10.48550/arXiv.astro-ph/0403420

Kaur, K., & Joshi, P. (2022, September 7). Fundamentals of Stellar Parameters Estimation through CMD of Star Clusters: Open (NGC2360) and Globular (NGC 5272). https://www.semanticscholar.org/paper/Fundamentals-of-Stellar-Parameters-Estimation-CMD-Kaur-Joshi/656c6282556d85c6d6fc8e4c4d95debf44a4f0a7?utm_source=consensus

Kawabata, T. (2018). Gaussian-input Gaussian mixture model for representing density maps and atomic models. Journal of Structural Biology, 203(1), 1–16. https://doi.org/10.1016/j.jsb.2018.03.002

Kerber, L. O., & Santiago, B. X. (2006). Mass segregation in rich LMC clusters from modelling of deep HST colour–magnitude diagrams. Astronomy & Astrophysics, 452(1), Article 1. https://doi.org/10.1051/0004-6361:20054198

K-Means clustering on the handwritten digits data using Scikit Learn in Python — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/k-means-clustering-on-the-handwritten-digits-data-using-scikit-learn-in-python/

KMeans — Scikit-learn 1.5.2 documentation. (n.d.). Retrieved December 4, 2024, from https://scikit-learn.org/1.5/modules/generated/sklearn.cluster.KMeans.html

Kodama, T., Arimoto, N., Barger, A., & Aragón-Salamanca, A. (1998). Astronomy and Astrophysics Evolution of the Colour-magnitude Relation of Early-type Galaxies in Distant Clusters. https://www.semanticscholar.org/paper/Astronomy-and-Astrophysics-Evolution-of-the-of-in-Kodama-Arimoto/63b13c59b29c8556c8d4def0bf142de7416c806d?utm_source=consensus

Koleva, M., Prugniel, Ph., Ocvirk, P., Le Borgne, D., & Soubiran, C. (2008). Spectroscopic ages and metallicities of stellar populations: Validation of full spectrum fitting. Monthly Notices of the Royal Astronomical Society, 385(4), 1998–2010. https://doi.org/10.1111/j.1365-2966.2008.12908.x

Kremer, J., Gieseke, F., Steenstrup Pedersen, K., & Igel, C. (2015). Nearest neighbor density ratio estimation for large-scale applications in astronomy. Astronomy and Computing, 12, 67–72. https://doi.org/10.1016/j.ascom.2015.06.005

Kremers, B. J. J., Ho, A., Citrin, J., & Plassche, K. L. van de. (2023). Two step clustering for data reduction combining DBSCAN and k-means clustering. Contributions to Plasma Physics, 63(5–6), e202200177. https://doi.org/10.1002/ctpp.202200177

Krone-Martins, A., & Moitinho, A. (2014). UPMASK: Unsupervised photometric membership assignment in stellar clusters. Astronomy & Astrophysics, 561, A57. https://doi.org/10.1051/0004-6361/201321143

Kumar, A. (2023, April 26). KMeans Silhouette Score Python Example. Analytics Yogi. https://vitalflux.com/kmeans-silhouette-score-explained-with-python-example/

Kurban, H., Kockan, C., Jenne, M., & Dalkilic, M. M. (2017a). Improving expectation maximization algorithm over stellar data. 2017 IEEE International Conference on Big Data (Big Data), 2559–2568. https://doi.org/10.1109/BigData.2017.8258215

Kurban, H., Kockan, C., Jenne, M., & Dalkilic, M. M. (2017b). Improving expectation maximization algorithm over stellar data. 2017 IEEE International Conference on Big Data (Big Data), 2559–2568. https://doi.org/10.1109/BigData.2017.8258215

Lang, D., Hogg, D. W., & Schlegel, D. J. (2014). WISE photometry for 400 million SDSS sources (No. arXiv:1410.7397). arXiv. https://doi.org/10.48550/arXiv.1410.7397

Lardo, C., Bellazzini, M., Pancino, E., Carretta, E., Bragaglia, A., & Dalessandro, E. (2011). Mining SDSS in search of multiple populations in globular clusters. Astronomy & Astrophysics, 525, A114. https://doi.org/10.1051/0004-6361/201015662

Li, J., & Nehorai, A. (2018). Gaussian mixture learning via adaptive hierarchical clustering. Signal Processing, 150, 116–121. https://doi.org/10.1016/j.sigpro.2018.04.013

Li, Z., & Han, Z. (2008). An isochrone data base and a rapid model for stellar population synthesis. Monthly Notices of the Royal Astronomical Society, 387(1), 105–114. https://doi.org/10.1111/j.1365-2966.2008.12793.x

Li, Z., Mao, C., & Chen, L. (2015). Explanation of a special color-magnitude diagram of star cluster NGC 1651 from different models (No. arXiv:1504.02563). arXiv. https://doi.org/10.48550/arXiv.1504.02563

Li, Z., Zhao, G., Zhang, R., Xue, X.-X., Chen, Y., & Amarante, J. A. S. (2023). Exploring the ex-situ components within Gaia DR3. Monthly Notices of the Royal Astronomical Society, 527(4), 9767–9781. https://doi.org/10.1093/mnras/stad3817

Lisboa-Wright, A. (2020). Modelling interstellar extinction in stellar populations [Masters, Liverpool John Moores University]. https://doi.org/10.24377/LJMU.t.00012848

Liu, H., Du, C., Ye, D., Zhang, J., & Deng, M. (2024, October 9). Exploration of Halo Substructures in IoM Space with \textit{Gaia} DR3. https://www.semanticscholar.org/paper/Exploration-of-Halo-Substructures-in-IoM-Space-with-Liu-Du/a2e40acc19351941d50001f94316af8689d3f186?utm_source=consensus

López-Cruz, O., Barkhouse, W. A., & Yee, H. K. C. (2004). The Color-Magnitude Effect in Early-Type Cluster Galaxies. The Astrophysical Journal, 614(2), 679. https://doi.org/10.1086/423664

Lu, H., Chen, J., Yan, K., Jin, Q., Xue, Y., & Gao, Z. (2017). A hybrid feature selection algorithm for gene expression data classification. Neurocomputing, 256, 56–62. https://doi.org/10.1016/j.neucom.2016.07.080

Lupton, R. H., Ivezic, Z., Gunn, J. E., Knapp, G., Strauss, M. A., & Yasuda, N. (2002). SDSS imaging pipelines. Survey and Other Telescope Technologies and Discoveries, 4836, 350–356. https://doi.org/10.1117/12.457307

Magoev, K., Krzhizhanovskaya, V. V., & Kovalchuk, S. V. (2018). Application of clustering methods for detecting critical acute coronary syndrome patients. Procedia Computer Science, 136, 370–379. https://doi.org/10.1016/j.procs.2018.08.277

Majaess, D., Turner, D. G., Gieren, W., & Ngeow, C. (2014). Evidence for photometric contamination in key observations of Cepheids in the benchmark galaxy IC 1613. Astronomy & Astrophysics, 572, A64. https://doi.org/10.1051/0004-6361/201424444

Majewski, S. R., Zasowski, G., & Nidever, D. L. (2011). LIFTING THE DUSTY VEIL WITH NEAR- AND MID-INFRARED PHOTOMETRY. I. DESCRIPTION AND APPLICATIONS OF THE RAYLEIGH–JEANS COLOR EXCESS METHOD. The Astrophysical Journal, 739(1), 25. https://doi.org/10.1088/0004-637X/739/1/25

Mapping the Galactic disk with the LAMOST and Gaia Red clump sample: VIII: Mapping the kinematics of the Galactic disk using mono-age and mono-abundance stellar populations. (n.d.). Retrieved November 29, 2024, from https://arxiv.org/html/2310.15408v3

March 2024, 07. (n.d.). Create a confusion matrix with Python. IBM Developer. Retrieved December 4, 2024, from https://developer.ibm.com/tutorials/awb-confusion-matrix-python/

Martínez-Galarza, J. R., Bianco, F., Crake, D., Tirumala, K., Mahabal, A. A., Graham, M. J., & Giles, D. (2021). A method for finding anomalous astronomical light curves and their analogs (No. arXiv:2009.06760). arXiv. https://doi.org/10.48550/arXiv.2009.06760

Masruroh, S. U., Putri, M. R., Hulliyah, K., Fiade, A., Aripiyanto, S., & Kharlie, A. T. (2023). Sentiment Analysis of Citizen on Twitter in Accessibility of Disabilities at The Public Space Using Support Vector Machine (SVM) Method with Radial Basis Function (RBF) Kernel. 2023 11th International Conference on Cyber and IT Service Management (CITSM), 1–6. https://doi.org/10.1109/CITSM60085.2023.10455689

Memetic Evolution of Training Sets with Adaptive Radial Basis Kernels for Support Vector Machines | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/9412495

Miettinen, O. (2018). Protostellar classification using supervised machine learning algorithms. Astrophysics and Space Science, 363(9), 197. https://doi.org/10.1007/s10509-018-3418-7

Mo, Z., & Siepel, A. (2023). Domain-adaptive neural networks improve supervised machine learning based on simulated population genetic data. PLOS Genetics, 19(11), e1011032. https://doi.org/10.1371/journal.pgen.1011032

Model Selection — Scikit-learn 1.5.2 documentation. (n.d.). Retrieved December 4, 2024, from https://scikit-learn.org/stable/auto_examples/model_selection/index.html

Mommert, M. (2017). PHOTOMETRYPIPELINE: An automated pipeline for calibrated photometry. Astronomy and Computing, 18, 47–53. https://doi.org/10.1016/j.ascom.2016.11.002

Montero, M. Á., Ruíz, R., García-Torres, M., & Sarro, L. M. (2010). Feature Selection Applied to Data from the Sloan Digital Sky Survey (N. García-Pedrajas, F. Herrera, C. Fyfe, J. M. Benítez, & M. Ali, Eds.; Vol. 6096, pp. 611–620). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-13022-9_61

Mousavian Anaraki, S. A., Haeri, A., & Moslehi, F. (2021). A hybrid reciprocal model of PCA and K-means with an innovative approach of considering sub-datasets for the improvement of K-means initialization and step-by-step labeling to create clusters with high interpretability. Pattern Analysis and Applications, 24(3), 1387–1402. https://doi.org/10.1007/s10044-021-00977-x

MuhammedYunus. (2023, August 14). Answer to “How to obtain specific principal components from PCA using sklearn or matplotlib, for EigenFaces?” [Online post]. Stack Overflow. https://stackoverflow.com/a/76899327

Multiple Stellar Populations: The evolutionary framework | Proceedings of the International Astronomical Union | Cambridge Core. (n.d.). Retrieved November 29, 2024, from https://www.cambridge.org/core/journals/proceedings-of-the-international-astronomical-union/article/multiple-stellar-populations-the-evolutionary-framework/DA94767804F0B3CB484C2260FCE932FB

Nag, K., & Pal, N. R. (2016). A Multiobjective Genetic Programming-Based Ensemble for Simultaneous Feature Selection and Classification. IEEE Transactions on Cybernetics, 46(2), 499–510. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2015.2404806

Nardiello, D., Milone, A. P., Piotto, G., Anderson, J., Bedin, L. R., Bellini, A., Cassisi, S., Libralato, M., & Marino, A. F. (2018). The Hubble Space Telescope UV Legacy Survey of Galactic Globular Clusters — XIV. Multiple stellar populations within M 15 and their radial distribution (No. arXiv:1803.05979). arXiv. https://doi.org/10.48550/arXiv.1803.05979

Nav. (2023, August 14). How to obtain specific principal components from PCA using sklearn or matplotlib, for EigenFaces? [Forum post]. Stack Overflow. https://stackoverflow.com/q/76899048

Ng, Y. K., Brogt, E., Chiosi, C., & Bertelli, G. (2002). Automatic observation rendering (AMORE): I. On a synthetic stellar population’s colour-magnitude diagram. Astronomy & Astrophysics, 392(3), 1129–1147. https://doi.org/10.1051/0004-6361:20020760

Niazmardi, S. (2024, September 8). Cluster-based Random Radial Basis Kernel Function for Hyperspectral Data Classification. https://www.semanticscholar.org/paper/Cluster-based-Random-Radial-Basis-Kernel-Function-Niazmardi/cae1a8e488264987f0ebf675e7f94a4e8bc20715?utm_source=consensus

Nidheesh, N., Abdul Nazeer, K. A., & Ameer, P. M. (2017). An enhanced deterministic K-Means clustering algorithm for cancer subtype prediction from gene expression data. Computers in Biology and Medicine, 91, 213–221. https://doi.org/10.1016/j.compbiomed.2017.10.014

Ordovás-Pascual, I., & Sánchez Almeida, J. (2014). A fast version of the k -means classification algorithm for astronomical applications. Astronomy & Astrophysics, 565, A53. https://doi.org/10.1051/0004-6361/201423806

Ou, X., Necib, L., & Frebel, A. (2022). Robust Clustering of the Local Milky Way Stellar Kinematic Substructures with Gaia eDR3 (No. arXiv:2208.01056). arXiv. https://doi.org/10.48550/arXiv.2208.01056

Pantoja, R., Catelan, M., Pichara, K., & Protopapas, P. (2022). Semi-Supervised Classification and Clustering Analysis for Variable Stars (No. arXiv:2209.09957). arXiv. https://doi.org/10.48550/arXiv.2209.09957

Pasquato, M., & Milone, A. (2019). Multiple Stellar Populations in NGC 2808: A Case Study for Cluster Analysis. arXiv: Solar and Stellar Astrophysics. https://www.semanticscholar.org/paper/Multiple-Stellar-Populations-in-NGC-2808%3A-a-Case-Pasquato-Milone/a9157a316a4054701e97cfe2db24ce4501c9a1f9?utm_source=consensus

PCA. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.decomposition.PCA.html

PCA Using Python: A Tutorial. (n.d.). Built In. Retrieved December 4, 2024, from https://builtin.com/machine-learning/pca-in-python

[PDF] CHARACTERISTIC PARAMETERS OF THE STELLAR POPULATIONS IN THE MILKY WAY GALAXY | Semantic Scholar. (n.d.). Retrieved November 27, 2024, from https://www.semanticscholar.org/paper/CHARACTERISTIC-PARAMETERS-OF-THE-STELLAR-IN-THE-WAY-Amado-Pinz%C3%B3n/e5b00e35faacf3917927a0ab6c69c6657a82f25d?utm_source=consensus

(PDF) Comparison of machine learning algorithms and feature importance analysis for star classification. (n.d.). Retrieved December 4, 2024, from https://www.researchgate.net/publication/377823985_Comparison_of_machine_learning_algorithms_and_feature_importance_analysis_for_star_classification

Peñarrubia, J., & Petersen, M. S. (2021). Identification of Sagittarius stream members in Angular Momentum space with Gaussian mixture techniques. Monthly Notices of the Royal Astronomical Society: Letters, 508(1), L26–L31. https://doi.org/10.1093/mnrasl/slab090

Plewa, P. M. (2018). Random Forest Classification of Stars in the Galactic Centre. Monthly Notices of the Royal Astronomical Society, 476(3), 3974–3980. https://doi.org/10.1093/mnras/sty511

Polsterer, K., Gieseke, F., Igel, C., & Goto, T. (2014, May 1). Improving the performance of photometric regression models via massive parallel feature selection. https://www.semanticscholar.org/paper/Improving-the-performance-of-photometric-regression-Polsterer-Gieseke/60bb23773b35824e7d064af5d69968040922884b?utm_source=consensus

Principal components analysis (PCA). (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/auto_examples/decomposition/plot_pca_3d.html

Prisinzano, L., Damiani, F., Sciortino, S., Flaccomio, E., Guarcello, M. G., Micela, G., Tognelli, E., Jeffries, R. D., & Alcalá, J. M. (2022). Low-mass young stars in the Milky Way unveiled by DBSCAN and Gaia EDR3: Mapping the star forming regions within 1.5 kpc. Astronomy & Astrophysics, 664, A175. https://doi.org/10.1051/0004-6361/202243580

Python Machine Learning — K-means. (n.d.). Retrieved December 4, 2024, from https://www.w3schools.com/python/python_ml_k-means.asp

Python — Scikit-learn random state in splitting dataset — Stack Overflow. (n.d.). Retrieved December 4, 2024, from https://stackoverflow.com/questions/42191717/scikit-learn-random-state-in-splitting-dataset

Raditya, M. H., Indwiarti, & Aniq Atiqi Rohmawati. (2022). House Prices Segmentation Using Gaussian Mixture Model-Based Clustering. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 6(5), 866–871. https://doi.org/10.29207/resti.v6i5.4459

Rahman, M., & Geiger, D. (2016). Quantum Clustering and Gaussian Mixtures. ArXiv. https://www.semanticscholar.org/paper/Quantum-Clustering-and-Gaussian-Mixtures-Rahman-Geiger/61bdcd042871a780647a989d6de327b824b3854e?utm_source=consensus

Ramos, P., Mateu, C., Antoja, T., Helmi, A., Castro-Ginard, A., Balbinot, E., & Carrasco, J. M. (2020). Full 5D characterisation of the Sagittarius stream with Gaia DR2 RR Lyrae. Astronomy & Astrophysics, 638, A104. https://doi.org/10.1051/0004-6361/202037819

Random Forest Classification with Scikit-Learn. (n.d.). Retrieved December 4, 2024, from https://www.datacamp.com/tutorial/random-forests-classifier-python

Random Forest Classifier using Scikit-learn — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/random-forest-classifier-using-scikit-learn/

RandomForestClassifier. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

Ree, C., Yoon, S.-J., Rey, S., & Lee, Y.-W. (2001). Synthetic Color-Magnitude Diagrams for omega Centauri and Other Massive Globular Clusters with Multiple Populations. arXiv: Astrophysics. https://www.semanticscholar.org/paper/Synthetic-Color-Magnitude-Diagrams-for-omega-and-Ree-Yoon/c782073ace1233138cb2ae5b5e81708099c410ed?utm_source=consensus

Revnivtsev, M., Van Den Berg, M., Burenin, R., Grindlay, J. E., Karasev, D., & Forman, W. (2010). Interstellar extinction and the distribution of stellar populations in the direction of the ultra-deep Chandra Galactic field. Astronomy and Astrophysics, 515, A49. https://doi.org/10.1051/0004-6361/200913527

Reynolds, D. (2009). Gaussian Mixture Models. In S. Z. Li & A. Jain (Eds.), Encyclopedia of Biometrics (pp. 659–663). Springer US. https://doi.org/10.1007/978-0-387-73003-5_196

Rhoads, J. E. (2000). Cosmic Ray Rejection by Linear Filtering of Single Images (No. arXiv:astro-ph/0002041). arXiv. https://doi.org/10.48550/arXiv.astro-ph/0002041

Risk Management for the Vera Rubin Observatory — Large Synoptic Survey Telescope | by Francia Riesco | Medium. (n.d.). Retrieved December 2, 2024, from https://fr4nc3.medium.com/risk-management-for-the-vera-rubin-observatory-large-synoptic-survey-telescope-cef9a249973f

Saito, R. K., Minniti, D., Dias, B., Hempel, M., Rejkuba, M., Alonso-García, J., Barbuy, B., Catelan, M., Emerson, J. P., Gonzalez, O. A., Lucas, P. W., & Zoccali, M. (2012). Milky Way demographics with the VVV survey — I. The 84-million star colour–magnitude diagram of the Galactic bulge. Astronomy & Astrophysics, 544, A147. https://doi.org/10.1051/0004-6361/201219448

Sanders, J. S., & Fabian, A. C. (2001). Adaptive binning of X-ray galaxy cluster images. Monthly Notices of the Royal Astronomical Society, 325(1), 178–186. https://doi.org/10.1046/j.1365-8711.2001.04410.x

Sanjaripour, S., Hemmati, S., Mobasher, B., Canalizo, G., Barish, B., Shivaei, I., Coil, A., Chartab, N., Jafariyazani, M., Reddy, N. A., & Azadi, M. (2024, October 9). Application of Manifold Learning to Selection of Different Galaxy Populations and Scaling Relation Analysis. https://www.semanticscholar.org/paper/Application-of-Manifold-Learning-to-Selection-of-Sanjaripour-Hemmati/691b2a1bcfc300c757aec63e4a98d6ab6443b533?utm_source=consensus

Santos, J. F. C., Bonatto, C., & Bica, E. (2005). Structure and stellar content analysis of the open cluster M 11 with 2MASS photometry. Astronomy & Astrophysics, 442(1), Article 1. https://doi.org/10.1051/0004-6361:20053378

Santos, J. F. C., Dottori, H., & Grosbøl, P. (2013). Properties of young star cluster systems: The age signature from near-infrared integrated colours. Astronomy and Astrophysics, 553. https://doi.org/10.1051/0004-6361/201220659

Schultheis, M., Chen, B. Q., Jiang, B. W., Gonzalez, O. A., Enokiya, R., Fukui, Y., Torii, K., Rejkuba, M., & Minniti, D. (2014). Mapping the Milky Way bulge at high resolution: The 3D dust extinction, CO, and X factor maps. Astronomy & Astrophysics, 566, A120. https://doi.org/10.1051/0004-6361/201322788

Scikit-learn SVM Tutorial with Python (Support Vector Machines) | DataCamp. (n.d.). Retrieved December 4, 2024, from https://www.datacamp.com/tutorial/svm-classification-scikit-learn-python

Scikit-learn/sklearn/cluster/_kmeans.py at main · scikit-learn/scikit-learn · GitHub. (n.d.). Retrieved December 4, 2024, from https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cluster/_kmeans.py

Scikit-learn/sklearn/metrics/_classification.py at main · scikit-learn/scikit-learn. (n.d.). GitHub. Retrieved December 4, 2024, from https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/_classification.py

SDSS data management and photometric quality assessment — Ivezić — 2004 — Astronomische Nachrichten — Wiley Online Library. (n.d.). Retrieved November 27, 2024, from https://onlinelibrary.wiley.com/doi/10.1002/asna.200410285

SDSS DR16 Data Analysis. (n.d.). Retrieved December 4, 2024, from https://kaggle.com/code/sanchitvj/sdss-dr16-data-analysis

SDSS Queries (astroquery.sdss) — Astroquery v0.4.8.dev342. (n.d.). Retrieved December 4, 2024, from https://astroquery.readthedocs.io/en/latest/sdss/sdss.html

SDSS-EPO/SciServer_notebooks/Week5_Galaxy_Zoo_clear.ipynb at master · brittlundgren/SDSS-EPO. (n.d.). GitHub. Retrieved December 4, 2024, from https://github.com/brittlundgren/SDSS-EPO/blob/master/SciServer_notebooks/Week5_Galaxy_Zoo_clear.ipynb

Searching for Pulsating Stars Using Clustering Algorithms | Proceedings of the International Astronomical Union | Cambridge Core. (n.d.). Retrieved December 1, 2024, from https://www.cambridge.org/core/journals/proceedings-of-the-international-astronomical-union/article/searching-for-pulsating-stars-using-clustering-algorithms/9BC55428D26A726C6971D8D492D971E3

Sentiment Analysis of Citizen on Twitter in Accessibility of Disabilities at The Public Space Using Support Vector Machine (SVM) Method with Radial Basis Function (RBF) Kernel | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved December 4, 2024, from https://ieeexplore.ieee.org/document/10455689

Shen, J., Bu, J., Ju, B., Jiang, T., Wu, H., & Li, L. (2012). Refining Gaussian mixture model based on enhanced manifold learning. Neurocomputing, 87, 19–25. https://doi.org/10.1016/j.neucom.2012.01.029

Shi, J.-H., Qiu, B., Luo, A.-L., He, Z.-D., Kong, X., & Jiang, X. (2022). A photometry pipeline for SDSS images based on convolutional neural networks. Monthly Notices of the Royal Astronomical Society, 516(1), 264–278. https://doi.org/10.1093/mnras/stac2144

Silhouette_score. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.metrics.silhouette_score.html

Sinaga, K. P., & Yang, M.-S. (2020). Unsupervised K-Means Clustering Algorithm. IEEE Access, 8, 80716–80727. IEEE Access. https://doi.org/10.1109/ACCESS.2020.2988796

Smale, L. F., Chantler, C. T., & Hudson, L. T. (2010). The effects of cosmic ray filtering on low intensity X-ray CCD data. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 619(1), 150–153. https://doi.org/10.1016/j.nima.2010.01.007

Souza, S. O., Kerber, L. O., Barbuy, B., Pérez-Villegas, A., Oliveira, R. A. P., & Nardiello, D. (2020). Self-consistent Analysis of Stellar Clusters: An Application to HST Data of the Halo Globular Cluster NGC 6752. The Astrophysical Journal, 890(1), 38. https://doi.org/10.3847/1538-4357/ab6a0f

Spatial Distribution of Evolved Giant Stars in the Galactic Disc Using DENIS Data | SpringerLink. (n.d.). Retrieved December 1, 2024, from https://link.springer.com/chapter/10.1007/978-94-011-5026-2_9

Sullivan, M., Treyer, M. A., Ellis, R. S., & Mobasher, B. (2004). An ultraviolet-selected galaxy redshift survey — III. Multicolour imaging and non-uniform star formation histories. Monthly Notices of the Royal Astronomical Society, 350(1), 21–34. https://doi.org/10.1111/j.1365-2966.2004.07649.x

SVC. (n.d.). Scikit-Learn. Retrieved December 4, 2024, from https://scikit-learn/stable/modules/generated/sklearn.svm.SVC.html

SVC Model in Python. To create a Support Vector Classifier… | by RandomResearchAI | Medium. (n.d.). Retrieved December 4, 2024, from https://randomresearchai.medium.com/svc-model-in-python-2d7b6d9434b4

Tamez Villarreal, J., & Barton, S. (2023). Stellar Classification based on Various Star Characteristics using Machine Learning Algorithms. Journal of Student Research, 12(1). https://doi.org/10.47611/jsrhs.v12i1.4375

The Milky Way Tomography with SDSS. II. Stellar Metallicity — Astrophysics Data System. (n.d.). Retrieved November 27, 2024, from https://ui.adsabs.harvard.edu/abs/2008ApJ...684..287I/abstract

The Seventh Data Release of the Sloan Digital Sky Survey — Astrophysics Data System. (n.d.). Retrieved November 27, 2024, from https://ui.adsabs.harvard.edu/abs/2009ApJS..182..543A/abstract

The Spatial Structure of Mono-abundance Sub-populations of the Milky Way Disk — Astrophysics Data System. (n.d.). Retrieved November 27, 2024, from https://ui.adsabs.harvard.edu/abs/2012ApJ...753..148B/abstract

UBV stellar photometry of bright stars in GC M5 — I. UV colour-magnitude and colour-colour diagrams and some peculiarities in the HB stellar distribution | Monthly Notices of the Royal Astronomical Society | Oxford Academic. (n.d.). Retrieved November 30, 2024, from https://academic.oup.com/mnras/article/326/1/102/1025750

Understanding Scikit-Learn’s SVC: Decision Function and Predict — GeeksforGeeks. (n.d.). Retrieved December 4, 2024, from https://www.geeksforgeeks.org/understanding-scikit-learns-svc-decision-function-and-predict/

Using StandardScaler() Function to Standardize Python Data | DigitalOcean. (n.d.). Retrieved December 4, 2024, from https://www.digitalocean.com/community/tutorials/standardscaler-function-in-python

Vilella-Rojo, G., Viironen, K., López-Sanjuan, C., Cenarro, A. J., Varela, J., Díaz-García, L. A., Cristóbal-Hornillos, D., Ederoclite, A., Marín-Franch, A., & Moles, M. (2015). Extracting H α flux from photometric data in the J-PLUS survey. Astronomy & Astrophysics, 580, A47. https://doi.org/10.1051/0004-6361/201526374

Wang, G.-Y., Wang, H.-F., Luo, Y.-P., Ting, Y.-S., Tepper-García, T., Bland-Hawthorn, J., & Carlin, J. (2024). Galactic-Seismology Substructures and Streams Hunter with LAMOST and Gaia. I. Methodology and Local Halo Results. The Astrophysical Journal, 974(2), 219. https://doi.org/10.3847/1538-4357/ad6d59

Xiao-Qing, W., & Jin-Meng, Y. (2021). Classification of star/galaxy/QSO and star spectral types from LAMOST data release 5 with machine learning approaches. Chinese Journal of Physics, 69, 303–311. https://doi.org/10.1016/j.cjph.2020.03.008

Xu, Q., Ding, C., Liu, J., & Luo, B. (2015). PCA-guided search for K-means. Pattern Recognition Letters, 54, 50–55. https://doi.org/10.1016/j.patrec.2014.11.017

Yekkehkhany, B., Safari, A., Homayouni, S., & Hasanlou, M. (2014). A COMPARISON STUDY OF DIFFERENT KERNEL FUNCTIONS FOR SVM-BASED CLASSIFICATION OF MULTI-TEMPORAL POLARIMETRY SAR DATA. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XL-2-W3, 281–285. WG II/1, WG II/4, ICWG II/IV, WG IV/7 </br> The 1st ISPRS International Conference on Geospatial Information Research (Volume XL-2/W3) — 15&ndash;17 November 2014, Tehran, Iran. https://doi.org/10.5194/isprsarchives-XL-2-W3-281-2014

York, D. G., Adelman, J., Anderson, J. E., Jr., Anderson, S. F., Annis, J., Bahcall, N. A., Bakken, J. A., Barkhouser, R., Bastian, S., Berman, E., Boroski, W. N., Bracker, S., Briegel, C., Briggs, J. W., Brinkmann, J., Brunner, R., Burles, S., Carey, L., Carr, M. A., … SDSS Collaboration. (2000). The Sloan Digital Sky Survey: Technical Summary. The Astronomical Journal, 120, 1579–1587. https://doi.org/10.1086/301513

Zhang, Y., Liu, H., & Deng, B. (2013). Evolutionary clustering with DBSCAN. 2013 Ninth International Conference on Natural Computation (ICNC), 923–928. https://doi.org/10.1109/ICNC.2013.6818108

Zhang, Y., & Zhao, Y. (2004). Automated clustering algorithms for classification of astronomical objects. Astronomy & Astrophysics, 422(3), 1113–1121. https://doi.org/10.1051/0004-6361:20040141

Zhu, H., Jiao, L., & Pan, J. (2006). Multi-population Genetic Algorithm for Feature Selection. In L. Jiao, L. Wang, X. Gao, J. Liu, & F. Wu (Eds.), Advances in Natural Computation (pp. 480–487). Springer. https://doi.org/10.1007/11881223_59

Zhu, J., Zhu, Z., Wang, C., & Ye, Z. (2009). Cosmic-Ray Detection Based on Gray-Scale Morphology of Spectroscopic CCD Images. Publications of the Astronomical Society of Australia, 26(1), 58–63. https://doi.org/10.1071/AS08050

Appendixes

A. Dataset query

See the Clean Photometry query to pick only objects with clean photometry from the SDSS imaging pipeline. Commonly, searching for the CLEAN flag suffices, but understanding the necessary circumstances for setting it is crucial in some situations (Lupton et al., 2001). The imaging pipeline adjusts the CLEAN flag depending on other flag values, which vary for stars and galaxies (Ivezić et al., 2004). To query star objects using PSF mags, utilize only primary objects and the flag combinations below. For the Star view, use the sample query to get only primary objects. Otherwise, add a “mode=1” constraint (Tucker et al., 2006). Perform the following tests, for instance, if we are interested in the r-band magnitudes of objects. If we are interested in various magnitudes or colors, add equivalent checks using AND for those bands. This query replaces human-readable flag names generated by flag functions with explicit values (Padmanabhan et al., 2007). Using explicit flag values streamlines query performance, ensuring more efficient and accurate results when working with large photometric datasets (Mommert, 2017).

SELECT TOP 50000 u,g,r,i,z,ra,dec, flags_r
FROM Star
WHERE
ra BETWEEN 180 and 181 AND dec BETWEEN -0.5 and 0.5
AND ((flags_r & 0x10000000) != 0)
 — detected in BINNED1
AND ((flags_r & 0x8100000c00a4) = 0)
 — not EDGE, NOPROFILE, PEAKCENTER, NOTCHECKED, PSF_FLUX_INTERP,
 — SATURATED, or BAD_COUNTS_ERROR
AND (((flags_r & 0x400000000000) = 0) or (psfmagerr_r <= 0.2))
 — not DEBLEND_NOPEAK or small PSF error
 — (substitute psfmagerr in other band as appropriate)
AND (((flags_r & 0x100000000000) = 0) or (flags_r & 0x1000) = 0)
 — not INTERP_CENTER or not COSMIC_RAY

B. Data Collection

import pandas as pd
file_path = ‘photometric_stars.csv’
data = pd.read_csv(file_path)
data.head()

data.describe(include=’all’)

data[[‘ra’, ‘dec’, ‘u’, ‘g’, ‘r’, ‘i’, ‘z’]].describe()

problematic_flags = [‘EDGE’, ‘NOPROFILE’, ‘SATURATED’, ‘COSMIC_RAY’, ‘INTERP_CENTER’]

available_flags = [flag for flag in problematic_flags if flag in data.columns]

filtered_data = data.copy()
for flag in available_flags:
if flag == ‘INTERP_CENTER’ and ‘COSMIC_RAY’ in available_flags:
filtered_data = filtered_data[
~((filtered_data[flag]) & (filtered_data[‘COSMIC_RAY’]))
]
else:
filtered_data = filtered_data[~filtered_data[flag]]

error_columns = [col for col in data.columns if ‘psfMagError’ in col]
filtered_data = filtered_data[
(filtered_data[error_columns] <= 0.2).all(axis=1)
]

original_count = data.shape[0]
filtered_count = filtered_data.shape[0]
excluded_count = original_count — filtered_count

print(f”Original dataset size: {original_count}”)
print(f”Filtered dataset size: {filtered_count}”)
print(f”Number of rows excluded: {excluded_count}”)
print(f”Percentage of data retained: {(filtered_count / original_count) * 100:.2f}%”)

C. Workflow

ra_range = (data[‘ra’].min(), data[‘ra’].max())
dec_range = (data[‘dec’].min(), data[‘dec’].max())

magnitude_columns = [‘psfMag_u’, ‘psfMag_g’, ‘psfMag_r’, ‘psfMag_i’, ‘psfMag_z’]
magnitude_summary = data[magnitude_columns].describe()

error_columns = [‘psfMagErr_u’, ‘psfMagErr_g’, ‘psfMagErr_r’, ‘psfMagErr_i’, ‘psfMagErr_z’]
psf_error_summary = data[error_columns].describe()

flag_columns = [‘EDGE’, ‘COSMIC_RAY’, ‘NOPROFILE’]
flag_counts = {flag: data[flag].sum() for flag in flag_columns if flag in data.columns}

total_rows = data.shape[0]
total_columns = data.shape[1]
critical_columns_present = all(col in data.columns for col in [‘ra’, ‘dec’, *magnitude_columns])

print(f”RA Range: {ra_range}”)
print(f”Dec Range: {dec_range}”)
print(“Photometric Magnitudes Summary:”)
print(magnitude_summary)

print(“PSF Magnitude Errors Summary:”)
print(psf_error_summary)

print(“Quality Flags Exclusion Counts:”)
print(flag_counts)

print(f”Total Rows: {total_rows}”)
print(f”Total Columns: {total_columns}”)
print(f”Critical Columns Present: {critical_columns_present}”)

D. Results

I. Data Preparation and Descriptive Statistics

import pandas as pd

# RA and Dec ranges
ra_min, ra_max = data[‘ra’].min(), data[‘ra’].max()
dec_min, dec_max = data[‘dec’].min(), data[‘dec’].max()

# Photometric Magnitudes
photometric_bands = [‘psfMag_u’, ‘psfMag_g’, ‘psfMag_r’, ‘psfMag_i’, ‘psfMag_z’]
photometric_stats = data[photometric_bands].describe()

# Dataset Retention
original_count = 5000
filtered_count = data.shape[0]
retention_rate = (filtered_count / original_count) * 100

# Validation Metrics — PSF Errors
psf_error_columns = [‘psfMagErr_u’, ‘psfMagErr_g’, ‘psfMagErr_r’, ‘psfMagErr_i’, ‘psfMagErr_z’]
psf_error_stats = data[psf_error_columns].describe()

print(“RA Range:”, (ra_min, ra_max))
print(“Dec Range:”, (dec_min, dec_max))
print(“Photometric Magnitude Statistics:”)
print(photometric_stats)
print(f”Dataset Retention: {filtered_count}/{original_count} ({retention_rate:.2f}%)”)
print(“PSF Magnitude Error Statistics:”)
print(psf_error_stats)

II. EDA

data[‘g-r’] = data[‘psfMag_g’] — data[‘psfMag_r’]

# Plot the CMD: (g-r) vs. r magnitude
plt.figure(figsize=(8, 6))
plt.scatter(data[‘g-r’], data[‘psfMag_r’], s=1, alpha=0.5, color=’blue’)
plt.gca().invert_yaxis() # Invert y-axis for magnitude (brighter stars lower)
plt.title(‘Color-Magnitude Diagram (g-r vs. r)’)
plt.xlabel(‘g-r (Color Index)’)
plt.ylabel(‘r (Magnitude)’)
plt.grid()
plt.show()

if ‘psfMag_r’ in data.columns and ‘psfMag_i’ in data.columns:
data[‘r-i’] = data[‘psfMag_r’] — data[‘psfMag_i’]
else:
print(“psfMag_r or psfMag_i column is missing!”)
plt.figure(figsize=(8, 6))
plt.scatter(data[‘r-i’], data[‘psfMag_i’], s=1, alpha=0.5, color=’green’)
plt.gca().invert_yaxis() # Invert y-axis for magnitude
plt.title(‘Color-Magnitude Diagram (r-i vs. i)’)
plt.xlabel(‘r-i (Color Index)’)
plt.ylabel(‘i (Magnitude)’)
plt.grid()
plt.show()

data[‘u-g’] = data[‘psfMag_u’] — data[‘psfMag_g’]
data[‘i-z’] = data[‘psfMag_i’] — data[‘psfMag_z’]

plt.figure(figsize=(8, 6))
plt.scatter(data[‘u-g’], data[‘psfMag_g’], s=1, alpha=0.5, color=’blue’)
plt.gca().invert_yaxis() # Invert y-axis for magnitude
plt.title(‘Color-Magnitude Diagram (u-g vs. g)’)
plt.xlabel(‘u-g (Color Index)’)
plt.ylabel(‘g (Magnitude)’)
plt.grid()
plt.show()

plt.figure(figsize=(8, 6))
plt.scatter(data[‘i-z’], data[‘psfMag_z’], s=1, alpha=0.5, color=’red’)
plt.gca().invert_yaxis() # Invert y-axis for magnitude
plt.title(‘Color-Magnitude Diagram (i-z vs. z)’)
plt.xlabel(‘i-z (Color Index)’)
plt.ylabel(‘z (Magnitude)’)
plt.grid()
plt.show()

magnitude_columns = [‘psfMag_u’, ‘psfMag_g’, ‘psfMag_r’, ‘psfMag_i’, ‘psfMag_z’]

histograms = {}
for mag in magnitude_columns:
plt.figure(figsize=(8, 6))
plt.hist(data[mag], bins=50, alpha=0.7, color=’blue’, edgecolor=’black’)
plt.title(f’Distribution of {mag}’)
plt.xlabel(‘Magnitude’)
plt.ylabel(‘Frequency’)
plt.grid()
plt.show()

histograms[mag] = {
“mean”: data[mag].mean(),
“median”: data[mag].median(),
“std_dev”: data[mag].std(),
“min”: data[mag].min(),
“max”: data[mag].max()
}

histograms

print(data[[‘g-r’, ‘r-i’]].describe())

# g-r
plt.figure(figsize=(8, 6))
plt.hist(data[‘g-r’], bins=50, alpha=0.7, color=’blue’, edgecolor=’black’)
plt.title(‘Histogram of g-r Color Index’)
plt.xlabel(‘g-r’)
plt.ylabel(‘Frequency’)
plt.grid()
plt.show()

# r-i
plt.figure(figsize=(8, 6))
plt.hist(data[‘r-i’], bins=50, alpha=0.7, color=’green’, edgecolor=’black’)
plt.title(‘Histogram of r-i Color Index’)
plt.xlabel(‘r-i’)
plt.ylabel(‘Frequency’)
plt.grid()
plt.show()

ra_dec_data = data[[‘ra’, ‘dec’]].dropna()

plt.figure(figsize=(10, 8))
density = gaussian_kde([ra_dec_data[‘ra’], ra_dec_data[‘dec’]])
density_values = density([ra_dec_data[‘ra’], ra_dec_data[‘dec’]])

plt.scatter(ra_dec_data[‘ra’], ra_dec_data[‘dec’], c=density_values, cmap=’viridis’, s=5, alpha=0.5)
plt.colorbar(label=’Density’)
plt.title(‘Scatter Plot with Density Overlay’)
plt.xlabel(‘RA (Degrees)’)
plt.ylabel(‘Dec (Degrees)’)
plt.grid()
plt.show()

plt.figure(figsize=(10, 6))
density_histogram, bins, _ = plt.hist(density_values, bins=30, color=’blue’, alpha=0.7, label=’Density’)
plt.title(‘Density Histogram’)
plt.xlabel(‘Density’)
plt.ylabel(‘Frequency’)
plt.grid()
plt.legend()
plt.show()

ra_bins = np.linspace(ra_dec_data[‘ra’].min(), ra_dec_data[‘ra’].max(), 20) # Adjust bin sizes as needed
dec_bins = np.linspace(ra_dec_data[‘dec’].min(), ra_dec_data[‘dec’].max(), 20)

grid_density, _, _ = np.histogram2d(ra_dec_data[‘ra’], ra_dec_data[‘dec’], bins=[ra_bins, dec_bins])

plt.figure(figsize=(10, 8))
plt.imshow(grid_density.T, extent=[ra_bins.min(), ra_bins.max(), dec_bins.min(), dec_bins.max()],
origin=’lower’, cmap=’plasma’, aspect=’auto’)
plt.colorbar(label=’Density (Stars per Unit Grid)’)
plt.title(‘Grid-Based Density Map’)
plt.xlabel(‘RA (Degrees)’)
plt.ylabel(‘Dec (Degrees)’)
plt.grid()
plt.show()

density_mean = density_values.mean()
density_median = np.median(density_values)
density_range = (density_values.min(), density_values.max())

print(f”Density Mean: {density_mean}”)
print(f”Density Median: {density_median}”)
print(f”Density Range: {density_range}”)
print(f”Grid Density Range: {grid_density.min()} to {grid_density.max()}”)

III. Clustering

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
import matplotlib.pyplot as plt
import pandas as pd

data[‘g-r’] = data[‘psfMag_g’] — data[‘psfMag_r’]
data[‘r-i’] = data[‘psfMag_r’] — data[‘psfMag_i’]

photometric_data = data[[‘g-r’, ‘r-i’]].dropna()

kmeans = KMeans(n_clusters=4, random_state=42)
data[‘KMeans_Labels’] = kmeans.fit_predict(photometric_data)

silhouette_avg = silhouette_score(photometric_data, data[‘KMeans_Labels’])

plt.figure(figsize=(10, 8))
for label in np.unique(data[‘KMeans_Labels’]):
cluster_data = photometric_data[data[‘KMeans_Labels’] == label]
plt.scatter(cluster_data[‘g-r’], cluster_data[‘r-i’], label=f’Cluster {label}’, alpha=0.7)

plt.title(‘K-Means Clustering: g-r vs r-i’)
plt.xlabel(‘g-r (Color Index)’)
plt.ylabel(‘r-i (Color Index)’)
plt.legend()
plt.grid()
plt.show()

print(f”Silhouette Score: {silhouette_avg}”)
print(“Cluster Centers:”)
print(kmeans.cluster_centers_)

from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from sklearn.metrics import silhouette_score

ra_dec_data = data[[‘ra’, ‘dec’]].dropna()

scaler = StandardScaler()
ra_dec_scaled = scaler.fit_transform(ra_dec_data)

dbscan = DBSCAN(eps=0.5, min_samples=10)
dbscan_labels = dbscan.fit_predict(ra_dec_scaled)
data[‘DBSCAN_Labels’] = dbscan_labels

if len(set(dbscan_labels)) > 1 and -1 not in set(dbscan_labels):
silhouette_dbscan = silhouette_score(ra_dec_scaled, dbscan_labels)
else:
silhouette_dbscan = None

plt.figure(figsize=(10, 8))
for label in np.unique(dbscan_labels):
cluster_data = ra_dec_data.iloc[dbscan_labels == label]
if label == -1:
plt.scatter(cluster_data[‘ra’], cluster_data[‘dec’], color=’red’, label=’Noise’, alpha=0.5)
else:
plt.scatter(cluster_data[‘ra’], cluster_data[‘dec’], label=f’Cluster {label}’, alpha=0.7)

plt.title(‘DBSCAN Clustering: RA vs Dec’)
plt.xlabel(‘RA (Degrees)’)
plt.ylabel(‘Dec (Degrees)’)
plt.legend()
plt.grid()
plt.show()

print(“Number of Clusters (excluding noise):”, len(set(dbscan_labels)) — (1 if -1 in dbscan_labels else 0))
print(“Number of Noise Points:”, (dbscan_labels == -1).sum())
if silhouette_dbscan:
print(“Silhouette Score:”, silhouette_dbscan)
else:
print(“Silhouette Score: Not Applicable (due to noise or single cluster)”)

ra_dec_data = data[[‘ra’, ‘dec’]].dropna()

kmeans_ra_dec = KMeans(n_clusters=4, random_state=42)
data[‘KMeans_RA_Dec_Labels’] = kmeans_ra_dec.fit_predict(ra_dec_data)

silhouette_ra_dec = silhouette_score(ra_dec_data, data[‘KMeans_RA_Dec_Labels’])

plt.figure(figsize=(10, 8))
for label in np.unique(data[‘KMeans_RA_Dec_Labels’]):
cluster_data = ra_dec_data[data[‘KMeans_RA_Dec_Labels’] == label]
plt.scatter(cluster_data[‘ra’], cluster_data[‘dec’], label=f’Cluster {label}’, alpha=0.7)

plt.title(‘K-Means Clustering: RA vs Dec’)
plt.xlabel(‘RA (Degrees)’)
plt.ylabel(‘Dec (Degrees)’)
plt.legend()
plt.grid()
plt.show()

print(f”Silhouette Score: {silhouette_ra_dec}”)
print(“Cluster Centers:”)
print(kmeans_ra_dec.cluster_centers_)

IV. Classification

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.metrics import classification_report, accuracy_score
from sklearn.preprocessing import StandardScaler

features = [‘g-r’, ‘u-g’, ‘r-i’, ‘ra’, ‘dec’]
data[‘g-r’] = data[‘psfMag_g’] — data[‘psfMag_r’]
data[‘u-g’] = data[‘psfMag_u’] — data[‘psfMag_g’]
data[‘r-i’] = data[‘psfMag_r’] — data[‘psfMag_i’]

data[‘population’] = np.where(data[‘g-r’] < 0.5, ‘white_dwarfs’,
np.where(data[‘g-r’] > 1.5, ‘giants’, ‘main_sequence’))

X = data[features]
y = data[‘population’]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

rf_classifier = RandomForestClassifier(n_estimators=100, random_state=42)
rf_classifier.fit(X_train, y_train)
rf_predictions = rf_classifier.predict(X_test)

rf_accuracy = accuracy_score(y_test, rf_predictions)
rf_classification_report = classification_report(y_test, rf_predictions, output_dict=True)
rf_feature_importances = rf_classifier.feature_importances_

svm_classifier = SVC(kernel=’rbf’, C=1, gamma=’scale’, random_state=42)
svm_classifier.fit(X_train_scaled, y_train)
svm_predictions = svm_classifier.predict(X_test_scaled)

svm_accuracy = accuracy_score(y_test, svm_predictions)
svm_classification_report = classification_report(y_test, svm_predictions, output_dict=True)

results = {
“Random Forest”: {
“Accuracy”: rf_accuracy,
“Classification Report”: rf_classification_report,
“Feature Importances”: dict(zip(features, rf_feature_importances))
},
“SVM”: {
“Accuracy”: svm_accuracy,
“Classification Report”: svm_classification_report
}
}

results

from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.svm import SVC
from sklearn.metrics import classification_report, accuracy_score, ConfusionMatrixDisplay

X = data[[‘g-r’, ‘u-g’, ‘r-i’, ‘ra’, ‘dec’]]
y = data[‘population’]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

param_grid = {
‘C’: [0.1, 1, 10, 100],
‘gamma’: [1, 0.1, 0.01, 0.001],
‘kernel’: [‘rbf’]
}
grid = GridSearchCV(SVC(), param_grid, refit=True, verbose=2, cv=5)
grid.fit(X_train, y_train)

print(“Best Parameters:”, grid.best_params_)
svm_model = grid.best_estimator_

y_pred = svm_model.predict(X_test)

print(“Accuracy:”, accuracy_score(y_test, y_pred))
print(“Classification Report:”)
print(classification_report(y_test, y_pred))

disp = ConfusionMatrixDisplay.from_estimator(
svm_model, X_test, y_test, display_labels=svm_model.classes_, cmap=’Blues’
)
disp.ax_.set_title(“Confusion Matrix: SVM Classification”)
disp.figure_.set_size_inches(8, 6)
disp.plot()

V. Validation

from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=3, random_state=42)
data[‘cluster_label’] = kmeans.fit_predict(data[[‘psfMag_g’, ‘psfMag_r’]])

isochrone_g = range(15, 26)
isochrone_r = [g — 0.4 * (g — 20) for g in isochrone_g]

plt.figure(figsize=(10, 7))
for label in data[‘cluster_label’].unique():
cluster_data = data[data[‘cluster_label’] == label]
plt.scatter(
cluster_data[‘psfMag_g’] — cluster_data[‘psfMag_r’], # g-r color index
cluster_data[‘psfMag_r’], # r magnitude
label=f’Cluster {label}’,
alpha=0.6,
s=15
)

plt.plot(
[g — r for g, r in zip(isochrone_g, isochrone_r)],
isochrone_r,
label=’Theoretical Isochrone’,
color=’red’,
linewidth=2
)

plt.gca().invert_yaxis()
plt.title(“CMD-Based Validation of Clusters”)
plt.xlabel(“g-r (Color Index)”)
plt.ylabel(“r (Magnitude)”)
plt.legend()
plt.grid(alpha=0.5)
plt.show()

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
coordinates = data[[‘ra’, ‘dec’]].dropna().values
pca_coordinates = pca.fit_transform(coordinates)

data[‘PCA1’] = pca_coordinates[:, 0]
data[‘PCA2’] = pca_coordinates[:, 1]

known_substructures = {
“Gaia-Enceladus”: [data[‘PCA1’].mean() — 0.5, data[‘PCA2’].mean() — 0.2],
“Sagittarius Stream”: [data[‘PCA1’].mean() + 0.3, data[‘PCA2’].mean() + 0.5],
“Thick Disk”: [data[‘PCA1’].mean(), data[‘PCA2’].mean()]
}

plt.figure(figsize=(10, 8))
plt.scatter(data[‘PCA1’], data[‘PCA2’], s=10, alpha=0.5, label=”Stellar Populations”)

# Add Galactic substructure markers
for name, coords in known_substructures.items():
plt.scatter(coords[0], coords[1], marker=’X’, s=200, label=name)

for name, coords in known_substructures.items():
plt.text(coords[0] + 0.1, coords[1] + 0.1, name, fontsize=10, color=’red’)

plt.xlabel(“PCA Component 1”)
plt.ylabel(“PCA Component 2”)
plt.title(“Comparison with Known Galactic Substructures”)
plt.legend()
plt.grid()
plt.show()

print(“Known Galactic Substructure PCA Coordinates:”)
for name, coords in known_substructures.items():
print(f”{name}: PCA1 = {coords[0]:.2f}, PCA2 = {coords[1]:.2f}”)

Introduction

Active Galactic Nuclei (AGN) rank among the brightest and most dynamic objects in the universe, with luminosities exceeding those of their host galaxies by factors of thousands (Urry and Padovani, 1995). These extraordinary phenomena are powered by the accretion of matter onto supermassive blackholes at galaxy centers. AGN variability occurs across timescales from minutes to decades, driven by complex interactions within the accretion disk, relativistic jets, and the corona. Their extreme energy output and intricate physics provide critical insights into the workings of the cosmos. This paper explores the current understanding of AGN, emphasizing how multi-wavelength and multi-messenger observations, coupled with advanced modeling techniques, have transformed our knowledge. We begin by examining the theoretical framework of AGN, particularly the unified model, which connects their observed diversity to orientation and obscuration effects. AGN classifications, such as Seyfert galaxies and blazars, offer valuable perspectives on relativistic jets and their extension (Smolčić et al., 2009) and intense X-ray and ultraviolet emissions from accretion disks (Urry and Padovani, 1995). Infrared studies have shown that dust is blocking views of the torus, and radio observations have helped us understand the shape of the jets (Smolčić et al., 2009), and X-ray studies have looked into the high-energy processes in the corona. Moreover, we review how high-energy neutrinos from blazars is one way that multi-messenger methods can help us learn more about AGN as cosmic accelerators (Keivani et al., 2018). Machine learning (ML) has made it possible to study variability trends and techniques that help to understand heavy blackholes. Important steps forward include taking pictures of the Event Horizon Telescope’s (EHT) shadow of M87’s supermassive blackhole, which shows how quickly this field is developing. And AGN properties with novel observing techniques and their important role in creating galaxy structures are all looked at in this work by putting all of these new developments together.

Figure 1: Unified Model. Credits: Urry and Padovani (1995).

Background and Theoretical Framework

AGNs are among the most powerful and luminous sources in the universe, with their observed diversity explained by the unified model. This model proposes a central structure comprising a supermassive blackhole, an accretion disk, relativistic jets, and an obscuring torus of gas and dust. The differences in observed AGN types—such as Seyfert galaxies, blazars, and radio galaxies—arise from the angle of observation and the extent of obscuration by the torus (Urry and Padovani, 1995). The accretion disk generates intense optical and ultraviolet radiation due to high-energy processes. A surrounding corona produces X-rays through inverse Compton scattering of disk photons. Magnetic fields in the accretion disk play a critical role in launching relativistic jets, which can extend up to hundreds of kiloparsecs. These jets emit synchrotron radiation that is detectable across radio and X-ray wavelengths, revealing the energetic dynamics of AGN (Keivani et al., 2018). Different viewing angles affect how AGNs are categorized. Due to an open viewing angle, Seyfert 1 galaxies show both wide and narrow emission lines, whereas Seyfert 2 galaxies only show narrow lines due to the torus obscuring the view. Due to relativistic beaming effects, blazars with jets that look directly at the viewer exhibit extreme light variability (Suh et al., 2019). Radio galaxies, on the other hand, have clear radio lobes and less changeable emissions, which means their jets are lined up at a wider angle from the viewer. Particularly around redshifts z ~ 2–3, that is referred to as cosmic noon, when star formation and blackhole growth peak, these processes play a crucial role in regulating the development of blackholes and galaxy architecture. A key probe into the dusty environment of AGN is provided by the dust surrounding the high-energy blackhole, which takes high-energy photons and re-emits them in infrared wavelengths (Suh et al., 2019). New theories show how important magnetohydrodynamic processes are in the formation of jets. These processes are driven by magnetic fields that take rotational momentum from the blackhole or accretion disk (Keivani et al., 2018). X-ray variability studies and echo maps, which can be seen in Figure 2, show that these models are based on facts. Long-term tracking has revealed changing-look AGN, which switch between Type 1 and Type 2 classifications, testing conventional classification methods and revealing the dynamic nature of AGN (Kocevski et al., 2022).

Figure 2: Morphological Classification. Credits: Urry and Padovani (1995).

Multi-Wavelength and Multi-Messenger Observations of AGN

AGN are extraordinary in their ability to emit energy across the entire electromagnetic spectrum, with each wavelength offering a unique window into the processes occurring near their central supermassive blackholes. Multi-wavelength observations and multi-messenger data enable a deeper understanding of AGN structures, dynamics, and the fundamental physical mechanisms, like radio observations that indicate relativistic jets and lobes’ hundreds-of-kiloparsec synchrotron radiation, like in Centaurus A that has 650,000-light-year jets (Urry and Padovani, 1995). This radio emission is necessary to map huge jet configurations and blackhole energy dissipation distant from the blackhole. Infrared observations probe that dusty torus around the accretion disk absorbs high-energy radiation and remits it at longer wavelengths. On lower wavelengths, optical and UV observations reveal accretion disk processes. Emission lines show gas velocities and compositions around the blackhole at these wavelengths. The accretion disk generates optical and UV thermal radiation at 10,000–20,000 K (Keivani et al., 2018). Heated corona and relativistic jets emit X-rays and gamma rays that are essential to the formation of high-energy phenomena. AGN jets and high-energy neutrino generation are linked by blazar neutrino observations like TXS 0506+056 observed by the IceCube Neutrino Observatory in 2017. These detections validate AGN as cosmic-ray accelerators and reveal particle interactions at relativistic speeds (Keivani et al., 2018). Laser Interferometer Space Antenna (LISA) detects gravitational waves from supermassive blackholes which might change our understanding of AGN formation and evolution (Danzmann, 1996). For instance, combined X-ray and optical observations show the energy transfer from the accretion disk to the corona, while simultaneous neutrino and gamma-ray data show the high-energy processes inside the accretion disk across timescales spanning from seconds to decades have improved because of these combined techniques. As Figure 3 shows, the light disk, corona, jets, and torus with their wavelengths and observing techniques.

Figure 3: AGN Emissions Across the Spectrum. Credits: Keivani, et al. (2018).

Data Analysis and Modeling Techniques

Advanced data analysis and modeling techniques are used to extract meaningful information from their highly variable and complex signals. By applying time-domain, spectral, and ML approaches, astronomers can uncover the physical processes driving AGN activity. Time-series analysis is essential for investigating AGN variability, in which researchers may discover transient events and distinctive timescales in AGN light curves using techniques like wavelet analysis and structure function analysis. Wavelet analysis, for instance, may offer light on dynamic processes in the accretion disk and jets by finding quasi-periodic oscillations and abrupt flares in AGN light curves. Cross-correlation analysis of light curves from various wavelengths shows time delays that match emitting region distances. This method has mapped energy transfer from the accretion disk to the corona using correlated optical and X-ray variability (Keivani et al., 2018). Spectral analysis helps researchers to find emission and absorption lines that reveal the temperature, composition, and velocity of the gas around the blackhole by breaking down the emitted light into its component wavelengths. UV spectra reveal accretion disk properties at temperatures of 10⁴–10⁵ K, whereas X-ray spectra reveal the dynamics of the hot corona at temperatures of 10⁹ K (Petropoulou et al., 2020). The high-energy processes inside the corona and relativistic jets are characterized by spectral fitting models such as power law and thermal comptonization. Power-law fits characterize jets’ non-thermal emission, whereas thermal comptonization models show how photons interact with hot electrons in the corona, revealing the emitting regions’ energy distribution. Reverberation mapping is a valuable tool for studying the structure and dynamics of the broad-line region (BLR) around the core blackhole. Astronomers may estimate BLR dimensions and blackhole masses by observing time delays between continuum emission and wide emission line shifts; this method has estimated blackhole masses from 10⁶ M☉ to 10¹⁰ M☉. ML is revolutionizing AGN research with forecast variability patterns, and evaluating big data sets from multi-wavelength surveys using algorithms such as supervised learning clustering that uses AGN light curves, ML models may detect changing-look AGN and forecast their future transitions (Kocevski et al., 2023). This method complements existing techniques, allowing researchers to examine large volumes of data and find hidden patterns. By simulating the behavior of magnetic fields in the accretion disk and jets, magnetohydrodynamic (MHD) models improve understanding of relativistic jets launched and collimated by magnetic fields as they take angular momentum from the disk, as shown in Figure 4 (Fausnaugh, 2017).

Figure 4: AGN Variability and Spectra. Credits: Fausnaugh, M.M. (2017).

Recent Advances and Case Studies

The study of AGN has advanced greatly during the last decade thanks to advances in observational techniques, modeling, and multi-messenger astronomy. These discoveries have improved our understanding of AGN phenomena and their effects on galaxy evolution and universe structure.

In 2017, the IceCube Neutrino Observatory detected high-energy neutrinos from the blazar TXS 0506+056, a major AGN finding. The first multi-messenger evidence connecting high-energy cosmic neutrinos to AGN, especially particle acceleration in relativistic jets, was verified. The neutrinos measured exceeded 10¹⁵ eV, revealing new insights into jet physics and cosmic ray sources (Keivani et al., 2018). Figure 5 shows how the interaction between jets and neutrino emissions is a key topic of AGN research, offering light on the high-energy processes at play in these extreme settings. Reverberation mapping has improved our understanding of AGN structure and dynamics, especially in BLR. The Sloan Digital Sky Survey Reverberation Mapping (SDSS-RM) program has tracked over 800 AGN and revised supermassive blackhole mass estimations by evaluating continuum emission fluctuation and broad-line response time delays. These observations have allowed for a better understanding of the connection between blackhole development and AGN activity (Fausnaugh, 2017). In 2019, the EHT captured the first blackhole shadow picture in galaxy M87, which is not an AGN, but this discovery allowed researchers to study AGN cores, especially ones with active accretion. These high-resolution observations challenge general relativity in severe gravitational fields and provide a foundation for future AGN imaging (Akiyama et al., 2021). Recent studies have also shown spectral characteristics switch between Type 1 and Type 2 classifications due to differences in accretion rates or tidal disruption events. AGN feedback governs star formation and expels gas from host galaxies via molecular outflows with velocities of 1000 km/s, according to ALMA observations. This feedback mechanism highlights AGN’s role in galaxy evolution. Multi-wavelength observations have improved our understanding of AGN jet dynamics. How energy propagates across jets and interacts with intergalactic material has been studied using radio, optical, and X-ray data from Centaurus A. The interdependence of jet dynamics and magnetic field structures has been shown by magnetic fields that have been measured to be as strong as 10⁻³ Gauss and have been found to be essential for accelerating and collimating jet particles.

Figure 5: Neutrino Emission and Jet Interaction. Credits: Keivani, et al. (2018).

Future Directions in AGN Research

Future breakthroughs, driven by advanced observatories and modeling techniques, will increase our understanding of these blazing cosmic engines and their effect on the evolution of the universe. Next-generation observatories like the JWST, SKA, and CTA will provide revolutionary AGN dynamics insights. JWST’s extraordinary infrared sensitivity will enable thorough studies of occluded AGN regions, revealing the dusty torus’s structure and accretion disk physical processes. JWST will investigate high-redshift AGN up to z >7, shedding light on the formation of the first supermassive blackholes and their evolution throughout the early stages of the universe (Villaescusa-Navarro et al., 2021). The SKA’s unequaled radio sensitivity will transform our understanding of AGN evolution and feedback mechanisms. With detailed mapping of jet structures and their interactions with intergalactic environments, it will give new insights into jet-environment dynamics. SKA’s ability to detect small emissions from accretion rings around blackholes would reveal jet collimation and particle acceleration (Urry and Padovani, 1995). To study particle acceleration in AGN jets, CTA will focus on high-energy gamma rays. Gamma-ray studies of blazars and similar phenomena will reveal extreme physical conditions in AGN systems (Branchesi, 2016). The merging of gravitational wave and neutrino astronomy will enable multimessenger AGN studies. LISA is planned to detect gravitational waves from supermassive blackhole mergers; these merger dynamics will be revealed over timescales of 10⁵ to 10⁶ years across blackhole and galaxy co-evolution, giving blackhole evolution rates (Keivani et al., 2018). By catching neutrinos with high energy associated with AGN jet processes, neutrino observatories like IceCube-Gen2 will detect over 10¹⁵ eV, which are required for proving cosmic ray acceleration and jet activity (Algaba et al., 2021). Advanced modeling techniques in magnetic fields’ in jet formation and accretion disk dynamics will be revealed via MHD simulations. Using them, we will classify AGN, uncover changing-look transitions, and analyze variability across varied populations (Petropoulou et al., 2020).

As Figure 6 shows, the SKA can map AGN jet dynamics, showing the SKA’s sensitivity to weak radio emissions and ability to discern complex jet structures, advancing our understanding of AGN physics.

Figure 6: Mapping AGN. Credits: Villaescusa-Navarro, et al. (2021).

Conclusion

Studies in high-energy astronomical events and how galaxies change over time. We examined AGN’s dynamic nature, ability to emit energy across the electromagnetic spectrum, and basic physical processes. These bright engines, powered by supermassive blackhole accretion, give insights into the interactions between blackholes and their surroundings. This highlighted the unified model’s role from Seyfert galaxies to blazars, which are oriented and obscured by relativistic jets, accretion disks, and dusty tori (Urry and Padovani, 1995). Moreover, classification of AGN demonstrates high theoretical breakthroughs that unify divergent observable properties under a single framework. In observations of multi-wavelength and multi-messenger, AGNs revealed previously unreachable insights into AGN physics. Observations across optical wavelengths have provided an understanding of AGN dynamics, from jet formation to accretion disk emission (Suh et al., 2019). The role of AGN in particle acceleration and blackhole mergers has been validated by multi-messenger observations of neutrinos and gravitational waves. Reverberation mapping and ML approaches have approximated blackhole masses from 10⁶ M☉ to 10¹⁰ M☉, and given insights into the hot corona with values up to 10⁹K and accretion disk with temperatures between 10⁴ K and 10⁵ K (Petropoulou et al., 2020). By linking observational data to theoretical models, we enhanced our understanding of AGN variability and structure. The EHT’s M87 blackhole shadow visualization and TXS 0506+056 high-energy neutrinos discovery (Keivani et al., 2018) opened new doors to how velocities of 1000 km/s influence interstellar medium dynamics, and observations of changing-look AGN and feedback dynamics reveal how AGNs impact their host galaxies. The future observatories will impact AGN evolution studies, and will explore jet dynamics, particle acceleration, and AGN’s role in high-redshift galaxies’ evolution. LISA and IceCube-Gen2 will also provide multi-messenger insights into black hole evolution and cosmic ray sources, resolving long-standing questions about AGN and their cosmic roles (Algaba et al., 2021).
Lastly, Figure 7 reveals AGN’s dynamic role in particle acceleration (Petropoulou et al., 2020). These studies show how AGN research connects black hole physics to the universe’s large-scale structure.

Figure 7: Multi-Epoch Modeling of Blazar TXS 0506+056. Credits: Petropoulou et al. (2020).

References

Pozzi, F., Vignali, C., Comastri, A., Bellocchi, E., Fritz, J., Gruppioni, C., Mignoli, M., Maiolino, R., Pozzetti, L., Brusa, M., Fiore, F. and Zamorani, G., 2010. The HELLAS2XMM survey: XIII. Multi-component analysis of the spectral energy distribution of obscured AGN. Astronomy & Astrophysics, 602, p.A2. Available at: https://doi.org/10.1051/0004-6361/201630223.

Fausnaugh, M.M., 2017. A new approach to the internal calibration of reverberation-mapping spectra. Publications of the Astronomical Society of the Pacific, 129(972), p.024007. Available at: https://doi.org/10.1088/1538-3873/129/972/024007.

Chainakun, P., Young, A.J. and Kara, E., 2016. Relativistic X-ray reverberation modelling of the combined time-averaged and lag-energy spectra in AGN. Monthly Notices of the Royal Astronomical Society, 460(3), pp.3076–3088. Available at: https://doi.org/10.1093/mnras/stw1105.

Keivani, A., Murase, K., Petropoulou, M., Fox, D.B., Cenko, S.B., Chaty, S., Coleiro, A., DeLaunay, J.J., Dimitrakoudis, S. and Evans, P.A., 2018. A multimessenger picture of the flaring blazar TXS 0506+056: Implications for high-energy neutrino emission and cosmic-ray acceleration. The Astrophysical Journal, 864(1), p.84. Available at: https://doi.org/10.3847/1538-4357/aad59a.

Urry, C.M. and Padovani, P., 1995. Unified schemes for radio-loud active galactic nuclei. Publications of the Astronomical Society of the Pacific, 107(715), p.803. Available at: https://doi.org/10.1086/133630.

Danzmann, K. and the LISA study team, 1996. LISA: Laser interferometer space antenna for gravitational wave measurements. Classical and Quantum Gravity, 13(11A), p.A247. Available at: https://doi.org/10.1088/0264-9381/13/11A/033.

Suh, H., Civano, F., Hasinger, G., Lusso, E., Marchesi, S., Schulze, A., Onodera, M., Rosario, D.J. and Sanders, D.B., 2019. Multi-wavelength properties of type 1 and type 2 AGN host galaxies in the Chandra-COSMOS Legacy Survey. The Astrophysical Journal, 872(2), p.168. Available at: https://doi.org/10.3847/1538-4357/ab01fb.

The EHT MWL Science Working Group, Algaba, J.C., Anczarski, J., Asada, K., Baloković, M., Chandra, S., Cui, Y.-Z., Falcone, A.D., Giroletti, M. and Goddi, C., 2021. Broadband multi-wavelength properties of M87 during the 2017 Event Horizon Telescope campaign. The Astrophysical Journal Letters, 911(1), p.L11. Available at: https://doi.org/10.3847/2041-8213/abef71.

Smolčić, V., Zamorani, G., Schinnerer, E., Bardelli, S., Bondi, M., Bîrzan, L., Carilli, C.L., Ciliegi, P., Elvis, M. and Impey, C.D., 2009. Cosmic evolution of radio-selected active galactic nuclei in the COSMOS field. The Astrophysical Journal, 696(1), p.24. Available at: https://doi.org/10.1088/0004-637X/696/1/24.

U, V., Lai, T., Bianchin, M., Remigio, R.P., Armus, L., Larson, K.L., Díaz-Santos, T., Evans, A., Stierwalt, S. and Law, D.R., 2022. GOALS-JWST: Resolving the circumnuclear gas dynamics in NGC 7469 in the mid-infrared. The Astrophysical Journal Letters, 940(1), p.L5. Available at: https://doi.org/10.3847/2041-8213/ac961c.

Satyapal, S., Kamal, L., Cann, J.M., Secrest, N.J. and Abel, N.P., 2021. The diagnostic potential of JWST in characterizing elusive AGNs. The Astrophysical Journal, 906(1), p.35. Available at: https://doi.org/10.3847/1538-4357/abbfaf.

Ni, Y., Genel, S., Anglés-Alcázar, D., Villaescusa-Navarro, F., Jo, Y., Bird, S., Di Matteo, T., Croft, R., Chen, N. and de Santi, N.S.M., 2023. The CAMELS project: Expanding the galaxy formation model space with new ASTRID and 28-parameter TNG and SIMBA suites. The Astrophysical Journal, 959(2), p.136. Available at: https://doi.org/10.3847/1538-4357/ad022a.

Villaescusa-Navarro, F., Anglés-Alcázar, D., Genel, S., Spergel, D.N., Somerville, R.S., Dave, R., Pillepich, A., Hernquist, L., Nelson, D. and Torrey, P., 2021. The CAMELS project: Cosmology and astrophysics with machine-learning simulations. The Astrophysical Journal, 915(1), p.71. Available at: https://doi.org/10.3847/1538-4357/abf7ba.

Branchesi, M., 2016. Multi-messenger astronomy: Gravitational waves, neutrinos, photons, and cosmic rays. Journal of Physics: Conference Series, 718(2), p.022004. Available at: https://doi.org/10.1088/1742-6596/718/2/022004.

Petropoulou, M., Murase, K., Santander, M., Buson, S., Tohuvavohu, A., Kawamuro, T., Vasilopoulos, G., Negoro, H., Ueda, Y. and Siegel, M.H., 2020. Multi-epoch modeling of TXS 0506+056 and implications for long-term high-energy neutrino emission. The Astrophysical Journal, 891(2), p.115. Available at: https://doi.org/10.3847/1538-4357/ab76d0.

Kocevski, D.D., Barro, G., McGrath, E.J., Finkelstein, S.L., Bagley, M.B., Ferguson, H.C., Jogee, S., Yang, G., Dickinson, M. and Hathi, N.P., 2023. CEERS Key Paper. II. A first look at the resolved host properties of AGN at 3 < z < 5 with JWST. The Astrophysical Journal Letters, 946(1), p.L14. Available at: https://doi.org/10.3847/2041-8213/acad00.

The Event Horizon Telescope Collaboration, Akiyama, K., Algaba, J.C., Alberdi, A., Alef, W., Anantua, R., Asada, K., Azulay, R., Baczko, A.-K. and Ball, D., 2021. First M87 Event Horizon Telescope results. VIII. Magnetic field structure near the event horizon. The Astrophysical Journal Letters, 910(1), p.L13. Available at: https://doi.org/10.3847/2041-8213/abe4de.

The ΛCDM model shows that regular matter makes up only a small part of the universe’s makeup. Most cosmic influence comprises enigmatic elements: dark matter, an invisible substance that exerts gravitational influence, and dark energy, a mysterious force that speeds up the universe’s expansion. This document examines evidence of these enigmatic elements’ existence and significant influence on the universe’s development. We will discuss the complex and convincing evidence supporting our understanding of these cosmic mysteries. This includes galactic rotation curves, CMB radiation, supernovae, and large-scale structures.

Introduction

The universe’s composition, evolution, and structure are explained with the Lambda Cold Dark Matter (ΛCDM) model (Szydłowski & Tambor, 2008). ΛCDM model describes that the universe starts with the Big Bang and that Ordinary (Baryonic) Matter (BM) constitutes only a tiny part of its constitution (Zhao et al., 2017). The rest is composed of enigmatic elements known as Dark Matter (DM), an invisible substance whose gravitational influence is essential for structure formation, and Dark Energy (DE), which controls the universe’s accelerated expansion (Motta et al., 2021). Proof for the ΛCDM model derives from various studies, including Cosmic Microwave Background (CMB) radiation. This document will discuss critical lines of evidence for DM and DE, exploring their influence on the universe and its evolution (Lesgourgues et al., 2015).

Overview

The ΛCDM model helps us to understand the universe. This model paints a picture of a universe composed of BM (5%), encompassing all visible elements. DM (27%), whose existence is deduced primarily through its gravitational influence on BM. DE (68%) is a mysterious energy that manages the universe’s accelerated expansion (Ade et al., 2013). CMB radiation, residues of the Big Bang, offers compelling support for the ΛCDM model. Moreover, intricate measurements of temperature and polarization fluctuations within the CMB have allowed scientists to determine critical properties of the universe with remarkable precision. This includes how CMB indicates a flat universe that matches the ΛCDM model’s predictions. Also, the fluctuations within the CMB reveal the relative proportions of BM, DM, and DE. By analyzing the CMB, scientists can trace how the universe has expanded over time, providing crucial insights into the influence of DM (Klypin et al., 2020). While the ΛCDM model has been successfully validated, we still have open questions and ongoing research. For example, the exact composition of DM remains a mystery. We are still searching for direct evidence of DM particles and exploring theoretical models explaining their properties. Researchers are still investigating whether DE is a constant property of spacetime or an evolving dynamic force. The ΛCDM model provides a framework for understanding the universe. As research continues, scientists aim to refine this model further, ultimately showing real the nature of the universe (Bullock & Boylan-Kolchin, 2017).

Dark Matter

It is an invisible element of the universe that exerts its influence through gravity in electromagnetic spectrums. Several studies corroborate the presence of DM. One is the Galactic Rotation Curves (GRC), which show that stars on the outskirts of galaxies move faster than visible light, implying the presence of DM (Ludwig, 2021). The CMB also reveals a lot about the nature of the universe. Its shifting temperatures and polarization patterns show that DM is crucial in shaping the cosmos (Li et al., 2008). GRC shows how DM influences the structure and history of the universe, whereas CMB observations show how it binds galaxies together. These various but connected pieces of evidence demonstrate DM’s pivotal role in the cosmos, connecting the observable universe to individual galaxies.

Galactic Rotation Curves

GRC presents fascinating proof of DM’s existence. These curves, which show how the orbital speeds of stars and gas vary with their proximity to a galaxy’s center, reveal an unexpected discrepancy. For example, the BM distribution and the laws of gravity and orbital speeds should decline when the distance from the galactic center increases. However, observations consistently show that GRC remains flat or rises at large radii. This proposes the existence of a massive, unseen element that exerts a dominant gravitational influence (Lisanti et al., 2018; Ludwig, 2021). This phenomenon is remarkably consistent across diverse galaxy types. Moreover, flat GRC in spiral galaxies offers the most direct proof for DM, but velocity dispersions and gravitational lensing effects in elliptical and irregular galaxies also support their existence (Rubin, 1993). This behavior across galaxies with varied star populations and evolution strengthens the case for DM’s pervasive presence (Stoehr et al., 2005).

Further evidence comes from the Tully-Fisher relation (TFR), that correlates spiral galaxies’s rotational velocities with their luminosity (Mo & Mao, 2000). The remarkable consistency of this relation across a wide range of galaxies implies a predictable link between visible mass and rotational speed. This link can only be explained by the gravitational influence of DM.

Finally, studies of galaxy clusters and their interactions offer additional support for DM. Moreover, DM’s existence reconciles discrepancies between the arrangement of hot gas and stars and the gravitational lensing in observations like the Bullet Cluster collision. Additionally, the separation of BM and DM in such collisions highlights DM’s non-interactive nature and its profound gravitational influence on cosmic structures (Harvey et al., 2015).

Cosmic Microwave Background

The CMB radiation, a Big Bang’s vestige, offers compelling evidence for DM. Precise measurements from missions like Planck and WMAP have yielded many insights that the CMB power spectrum is significant, as it reveals that the overall density of matter includes DM. The CMB peaks reflect oscillations in the dense, hot matter that filled the early universe. DM’s gravitational interactions determined the characteristics of these oscillations (Kelso et al., 2013). Moreover, CMB polarization provides further support for DM. A tiny part of the CMB is polarized caused by Thomson’s separation of electrons at the surface of the last scattering. The detection of E-mode (even-parity) polarization strongly confirms the acoustic oscillations in the spectrum’s temperature, providing an independent line of evidence for DM (Cyr-Racine et al., 2013). These polarization measurements complement the temperature data, offering a view through inside the early universe’s circumstances and DM’s quantity constraints. CMB photons’ gravitational lensing also points to the presence of DM. As CMB photons traverse the universe, their paths are subtly distorted by the gravitational pull of matter, including DM. These distortions help to plot the dissemination of all matter along CMB, revealing the large-scale gravitational influence of DM and providing an additional probe of its existence (Mellier, 2010). Also, CMB offers insights into DM’s role during reionization. During ionizing photon production, it is influenced by the gravitational clustering of both dark and baryonic matter. This indirect probe complements direct measurements of matter density fluctuations, helping us understand DM and cosmic structure formation (Fukugita & Kawasaki, 1994). Furthermore, CMB data with large-scale galaxy surveys and Lyman-alpha forest observations helps map DM distribution and behavior across cosmic history (Rauch, 1998). The CMB’s large-scale pattern is consistent with the ΛCDM model, highlighting DM’s role in the universe.

Lastly, these perspectives reinforce one another as they support the existence of DM across different cosmic epochs and scales. GRCs underscore the ongoing need for DM to explain galactic dynamics in the present universe. CMB offers independent evidence from the universe’s infancy, demonstrating DM’s influence on its early development. Both GRCs and the CMB provide complementary evidence for DM as a fundamental component of our cosmological model (Sofue & Rubin, 2001).

Dark Energy

DE is a mysterious force that fills the universe and is responsible for its accelerated expansion. Observations of Supernovae Type Ia (SNe Ia) consistent luminosity reveal that distant galaxies are receding from us at an ever-increasing rate. This phenomenon is directly attributed to DE (Sullivan, 2010). Additionally, CMB provides compelling evidence of DE because it is characterized by a flat geometry and precise temperature fluctuations, necessitating a significant amount of DE to align with observations. Furthermore, the CMB’s Integrated Sachs-Wolfe effect, which traces how cosmic photons interact with evolving gravitational potentials over time, supports DE’s influence (Planck Collaboration et al., 2015). These distinct yet interconnected pieces of evidence solidify DE’s role in shaping our universe’s evolution.

Supernovae Type Ia Observation

SNe Ia provides the most convincing confirmation for DE, the enigmatic force propelling the universe’s accelerated expansion. Due to their consistent peak brightness, these supernovae act as “standard candles,” allowing astronomers to accurately gauge distances to far-off galaxies. In the late 1990s, observations revealed that remote SNe Ia faded more than expected in a decelerating universe, indicating the presence of DE (Davis & Parkinson, 2016). Additionally, ongoing studies show that SNe Ia data is at various redshifts. This expanded dataset strengthens cosmological conclusions and allows us to trace the universe’s expansion history over time. By comparing observations with different DE models, including a cosmological constant (Λ), astronomers are honing their understanding of this mysterious force (Wang & Garnavich, 2001). Interpreting SNe Ia data presents challenges, such as variations in their intrinsic luminosity, potential evolution over cosmic time, and the effects of interstellar dust. Researchers carefully calibrate and correct for these factors to ensure precise distance measurements. Cross-referencing SNe Ia data with other cosmological probes helps minimize systematic errors and provides a more robust picture of how DE influences the universe’s expansion (Brout & Scolnic, 2020). SNe Ia observations have also spurred investigations into alternative DE models and potential modifications to General Relativity. While a cosmological constant remains the easy solution for DE, some models propose a dynamic DE that evolves or modifies our understanding of gravity over time — determining whether DE is constant or changes over time will have profound implications for the ultimate universe’s destiny (Bento et al., 2004). This means astronomers continue refining their techniques and incorporating data from new surveys to unravel this cosmic mystery and understand our universe’s fundamental laws.

Cosmic Microwave Background

The CMB radiation also provides DE’s existing evidence. Precise measurements by Planck and WMAP missions have revealed the geometry of the universe, its expansion journey, and composition (Corasaniti & Melchiorri, 2007). The CMB’s first apparent scale size of the acoustic peak indicates a flat universe, a geometry that necessitates a significant contribution from both DM and DE. This geometric constraint is fundamental to show how DE influences the universe’s accelerated expansion (Ade et al., 2013). Also, the Integrated Sachs-Wolfe (ISW) further supports DE. As CMB photons traverse evolving gravitational potentials, they gain or lose energy, reflecting DE’s influence on the fabric of spacetime. This way, correlating the ISW effect with large-scale structure surveys reinforces the connection between DE and the universe’s accelerating expansion (Corasaniti et al., 2005). Moreover, the CMB’s intricate patterns of temperature and polarization fluctuations place tight constraints on the properties of DE. For this reason, theoretical models must reproduce CMB observations, including the universe’s age, the details of recombination, and the growth rate of cosmic structures (Hu, 2002). These constraints refine our understanding of DE within the ΛCDM model and allow us to test its consistency (Park & Ratra, 2017). Besides, CMB observations have also inspired investigations into non-dynamic DE modes, where density evolves, and potential modifications to General Relativity. Such models often involve scalar fields or changes to the fundamental laws of gravity. They offer a different solution for cosmic acceleration and the observed structure of the CMB, possibly hinting at new physics beyond our current understanding (Chamseddine & Mukhanov, 2016). Ultimately, the CMB plays a role in modern cosmology, offering multiple lines of evidence for DE. By studying CMB’s properties and cross-correlating with other cosmological evidence, we better understand the universe’s composition and evolution (Boughn & Crittenden, 2004).

Growth of Structure

The interplay of DM and BM fluctuations following the Big Bang gives us the galaxies we watch today. Throughout the rapid expansion of the early universe, quantum fluctuations were amplified into cosmic-scale density variations. These overdensities, dominated by the gravitational influence of DM, became the seeds of all subsequent structure formation (Tenkanen, 2019). Unlike BM, DM interacts primarily through gravity, allowing it to cluster freely and form the initial gravitational wells across the cosmos. Numerical simulations illustrate this process, revealing a hierarchical buildup of dark matter structures. The intricate cosmic web, composed of filaments, voids, and dense halo nodes, emerged from these early fluctuations (Springel et al., 2005). Moreover, recent research suggests that the precise essence of DM, whether cold, warm, or self-interacting, could subtly influence the formation of the tiniest dwarf galaxies and the DM distribution inside galaxy clusters (Gnedin et al., 2001). When interacting, DM began to cluster around these early fluctuations, primarily through gravity. As the universe cooled, the baryonic matter eventually fell into the gravitational wells these expanding DM halos created. This marked the beginning of the “Dark Ages,” where the first structures formed without stars. Within the structure of the cold DM model, DM continued to collapse under gravity, creating a hierarchically organized network of halos, filaments, and voids. This model successfully predicts the universe’s large-scale structure (Blumenthal et al., 1984). In another case, Baryonic matter, with its complex interactions with radiation, followed a more intricate path. Once recombination allows radiation to flow freely, baryons could collapse into DM potential wells. The star formation, gas cooling, and processes from massive and supermassive black holes complex physics shaped the galaxies’s evolution (Navarro et al.,1994). These processes regulated the distribution of baryonic matter within DM halos, ultimately giving rise to a diverse range of galaxy types and morphologies (Tseliakhovich & Hirata, 2010). Moreover, DM and BM produced further complexity on more minor scales. For instance, Baryonic processes can reshape DM halos and reactions from active galactic nuclei or star formation, and they can even expel baryons from smaller halos, suppressing the formation of visible galaxies inside the minor DM structures (Fan et al., 2013). Lastly, the fluctuations of the beginning density seeded by inflation, the cosmic transformation highlights the relationship between DM, which provides gravitational scaffolding, and baryonic physics, which shapes cosmic structures. They offer a comprehensive understanding of how the universe transformed from its smooth beginnings to the complex structure that we observe today (Huterer et al., 2013)

Conclusions

As the standard cosmological model affirmed, DM and DE shape the universe’s structure despite their enigmatic nature. Ongoing research and new techniques will refine our understanding of these components. Future investigations, including CMB experiments, deep-sky surveys, and particle physics experiments, solidify our current knowledge or expand our cosmological paradigm (Delabrouille et al., 2017). Moreover, the evidence for DM, including galactic rotation curves, gravitational lensing, and others, strongly supports its existence. Nevertheless, we still don’t know the primary nature of DM’s particles. Direct detection experiments promise to unveil these elusive particles’ properties (Undagoitia and Rauch, 2015). Similarly, DE presents an enigmatic puzzle in the universe’s accelerated expansion. Its repulsive force, counteracting gravity on vast scales, suggests theories beyond the ΛCDM model. Missions like the Euclid satellite and the Vera C. Rubin Observatory will probe DE’s characteristics and role in cosmic evolution by mapping the known universe (Laureijs et al., 2012; Sheldon et al., 2023). Ultimately, understanding DM and DE is an aspect of present-day science that will reshape our understanding of the cosmos (Dine, 2001). Beyond academic interest, unraveling these mysteries will address questions about the universe’s origin, evolution, and destiny. The coming decades promise to be a golden age of cosmological discovery, revealing insights into the secrets of the universe and our place within it (Frieman et al., 2008).

References

2005nfcd.conf..267Y Page 267. (n.d.). Retrieved April 11, 2024, from https://adsabs.harvard.edu/full/2005nfcd.conf..267Y

Abbott, T. M. C., Allam, S., Andersen, P., Angus, C., Asorey, J., Avelino, A., Avila, S., Bassett, B. A., Bechtol, K., Bernstein, G. M., Bertin, E., Brooks, D., Brout, D., Brown, P., Burke, D. L., Calcino, J., Rosell, A. C., Carollo, D., Kind, M. C., … Collaboration), (DES. (2019). First Cosmology Results using Type Ia Supernovae from the Dark Energy Survey: Constraints on Cosmological Parameters. The Astrophysical Journal Letters, 872(2), L30. https://doi.org/10.3847/2041-8213/ab04fa

Abramo, L. R., Finelli, F., & Pereira, T. S. (2004). Constraining Born-Infeld models of dark energy with CMB anisotropies. Physical Review D, 70(6), 063517. https://doi.org/10.1103/PhysRevD.70.063517

Adachi, S., Faúndez, M. A. O. A., Arnold, K., Baccigalupi, C., Barron, D., Beck, D., Bianchini, F., Chapman, S., Cheung, K., Chinone, Y., Crowley, K., Dobbs, M., Bouhargani, H. E., Elleflot, T., Errard, J., Fabbian, G., Feng, C., Fujino, T., Galitzki, N., … Collaboration), (The Polarbear. (2020). A Measurement of the CMB E-mode Angular Power Spectrum at Subdegree Scales from 670 Square Degrees of POLARBEAR Data. The Astrophysical Journal, 904(1), 65. https://doi.org/10.3847/1538-4357/abbacd

Amendola, L., Appleby, S., Bacon, D., Baker, T., Baldi, M., Bartolo, N., Blanchard, A., Bonvin, C., Borgani, S., Branchini, E., Burrage, C., Camera, S., Carbone, C., Casarini, L., Cropper, M., de Rham, C., Di Porto, C., Ealet, A., Ferreira, P. G., … The Euclid Theory Working Group. (2013). Cosmology and Fundamental Physics with the Euclid Satellite. Living Reviews in Relativity, 16(1), 6. https://doi.org/10.12942/lrr-2013-6

An excess of small-scale gravitational lenses observed in galaxy clusters | Science. (n.d.). Retrieved April 11, 2024, from https://www.science.org/doi/10.1126/science.aax5164

Andrew Howell, D., Sullivan, M., Nugent, P. E., Ellis, R. S., Conley, A. J., Le Borgne, D., Carlberg, R. G., Guy, J., Balam, D., Basa, S., Fouchez, D., Hook, I. M., Hsiao, E. Y., Neill, J. D., Pain, R., Perrett, K. M., & Pritchet, C. J. (2006). The type Ia supernova SNLS-03D3bb from a super-Chandrasekhar-mass white dwarf star. Nature, 443(7109), 308–311. https://doi.org/10.1038/nature05103

Aslanyan, G., Price, L. C., Adams, J., Bringmann, T., Clark, H. A., Easther, R., Lewis, G. F., & Scott, P. (2016). Ultracompact Minihalos as Probes of Inflationary Cosmology. Physical Review Letters, 117(14), 141102. https://doi.org/10.1103/PhysRevLett.117.141102

Baltay, C. (2014). The accelerating universe and dark energy. International Journal of Modern Physics D, 23(06), 1430012. https://doi.org/10.1142/S0218271814300122

Basilakos, S., & Lima, J. A. S. (2010). Constraints on cold dark matter accelerating cosmologies and cluster formation. Physical Review D, 82(2), 023504. https://doi.org/10.1103/PhysRevD.82.023504

Batista, R. C., & Pace, F. (2013). Structure formation in inhomogeneous Early Dark Energy models. https://repositorio.ufrn.br/handle/123456789/30835

Battaner, E., & Florido, E. (2000). The rotation curve of spiral galaxies and its cosmological implications (arXiv:astro-ph/0010475). arXiv. https://doi.org/10.48550/arXiv.astro-ph/0010475

Battaner, E., Garrido, J. L., Membrado, M., & Florido, E. (1992). Magnetic fields as an alternative explanation for the rotation curves of spiral galaxies. Nature, 360(6405), 652–653. https://doi.org/10.1038/360652a0

Bennett, C. L., Larson, D., Weiland, J. L., Jarosik, N., Hinshaw, G., Odegard, N., Smith, K. M., Hill, R. S., Gold, B., Halpern, M., Komatsu, E., Nolta, M. R., Page, L., Spergel, D. N., Wollack, E., Dunkley, J., Kogut, A., Limon, M., Meyer, S. S., … Wright, E. L. (2013). Nine-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Final Maps and Results. The Astrophysical Journal Supplement Series, 208(2), 20. https://doi.org/10.1088/0067-0049/208/2/20

Bento, M. C., Bertolami, O., Santos, N. M. C., & Sen, A. A. (2005). Supernovae constraints on models of dark energy reexamined. Physical Review D, 71(6), 063501. https://doi.org/10.1103/PhysRevD.71.063501

Bertone, G., & Hooper, D. (2018). A History of Dark Matter. Reviews of Modern Physics, 90(4), 045002. https://doi.org/10.1103/RevModPhys.90.045002

Blondin, S., Matheson, T., Kirshner, R. P., Mandel, K. S., Berlind, P., Calkins, M., Challis, P., Garnavich, P. M., Jha, S. W., Modjaz, M., Riess, A. G., & Schmidt, B. P. (2012). THE SPECTROSCOPIC DIVERSITY OF TYPE Ia SUPERNOVAE*. The Astronomical Journal, 143(5), 126. https://doi.org/10.1088/0004-6256/143/5/126

Blumenthal, G. R., Faber, S. M., Primack, J. R., & Rees, M. J. (1984). Formation of galaxies and large-scale structure with cold dark matter. Nature, 311(5986), 517–525. https://doi.org/10.1038/311517a0

Bode, P., Ostriker, J. P., & Turok, N. (2001). Halo Formation in Warm Dark Matter Models. The Astrophysical Journal, 556(1), 93. https://doi.org/10.1086/321541

Bottema, R., & Pestaña, J. L. G. (2015). The distribution of dark and luminous matter inferred from extended rotation curves. Monthly Notices of the Royal Astronomical Society, 448(3), 2566–2593. https://doi.org/10.1093/mnras/stv182

Boughn, S., & Crittenden, R. (2004). A correlation between the cosmic microwave background and large-scale structure in the Universe. Nature, 427(6969), 45–47. https://doi.org/10.1038/nature02139

Bozorgnia, N., & Schwetz, T. (2014). What is the probability that direct detection experiments have observed dark matter? Journal of Cosmology and Astroparticle Physics, 2014(12), 015. https://doi.org/10.1088/1475-7516/2014/12/015

Bradač, M., Allen, S. W., Treu, T., Ebeling, H., Massey, R., Morris, R. G., Linden, A. von der, & Applegate, D. (2008). Revealing the Properties of Dark Matter in the Merging Cluster MACS J0025.4–1222*. The Astrophysical Journal, 687(2), 959. https://doi.org/10.1086/591246

Brandt, T. D. (2016). CONSTRAINTS ON MACHO DARK MATTER FROM COMPACT S℡LAR SYSTEMS IN ULTRA-FAINT DWARF GALAXIES. The Astrophysical Journal Letters, 824(2), L31. https://doi.org/10.3847/2041-8205/824/2/L31

Brout, D., & Scolnic, D. (2021). It’s Dust: Solving the Mysteries of the Intrinsic Scatter and Host-galaxy Dependence of Standardized Type Ia Supernova Brightnesses. The Astrophysical Journal, 909(1), 26. https://doi.org/10.3847/1538-4357/abd69b

Buckley, M. R., & DiFranzo, A. (2018). Collapsed Dark Matter Structures. Physical Review Letters, 120(5), 051102. https://doi.org/10.1103/PhysRevLett.120.051102

Burns, C. R., Parent, E., Phillips, M. M., Stritzinger, M., Krisciunas, K., Suntzeff, N. B., Hsiao, E. Y., Contreras, C., Anais, J., Boldt, L., Busta, L., Campillay, A., Castellón, S., Folatelli, G., Freedman, W. L., González, C., Hamuy, M., Heoflich, P., Krzeminski, W., … Torres, S. (2018). The Carnegie Supernova Project: Absolute Calibration and the Hubble Constant. The Astrophysical Journal, 869(1), 56. https://doi.org/10.3847/1538-4357/aae51c

Campanelli, L., Cea, P., Fogli, G. L., & Marrone, A. (2011). Testing the isotropy of the Universe with type Ia supernovae. Physical Review D, 83(10), 103503. https://doi.org/10.1103/PhysRevD.83.103503

Capozziello, S., Cardone, V. F., & Troisi, A. (2007). Low surface brightness galaxy rotation curves in the low energy limit of Rn gravity: No need for dark matter?: LSB rotation curves and Rn gravity theories. Monthly Notices of the Royal Astronomical Society, 375(4), 1423–1440. https://doi.org/10.1111/j.1365-2966.2007.11401.x

Chamseddine, A. H., & Mukhanov, V. (2016). Inhomogeneous dark energy. Journal of Cosmology and Astroparticle Physics, 2016(02), 040. https://doi.org/10.1088/1475-7516/2016/02/040

Chan, M. H. (2019). A universal constant for dark matter-baryon interplay. Scientific Reports, 9(1), 3570. https://doi.org/10.1038/s41598-019-39717-x

Chotard, N., Gangler, E., Aldering, G., Antilogus, P., Aragon, C., Bailey, S., Baltay, C., Bongard, S., Buton, C., Canto, A., Childress, M., Copin, Y., Fakhouri, H. K., Hsiao, E. Y., Kerschhaggl, M., Kowalski, M., Loken, S., Nugent, P., Paech, K., … Wu, C. (2011). The reddening law of type Ia supernovae: Separating intrinsic variability from dust using equivalent widths. Astronomy & Astrophysics, 529, L4. https://doi.org/10.1051/0004-6361/201116723

Clowe, D., Bradač, M., Gonzalez, A. H., Markevitch, M., Randall, S. W., Jones, C., & Zaritsky, D. (2006). A Direct Empirical Proof of the Existence of Dark Matter*. The Astrophysical Journal, 648(2), L109. https://doi.org/10.1086/508162

CMB temperature and polarization anisotropy fundamentals — ScienceDirect. (n.d.). Retrieved April 12, 2024, from https://www.sciencedirect.com/science/article/abs/pii/S0003491602000222?via%3Dihub

Cmbcorrelations / The late-time integrated Sachs Wolfe effect. (n.d.). Retrieved April 12, 2024, from http://cmbcorrelations.pbworks.com/w/page/4563978/The%20late-time%20integrated%20Sachs%20Wolfe%20effect

Conley, A., Carlberg, R. G., Guy, J., Howell, D. A., Jha, S., Riess, A. G., & Sullivan, M. (2007). Is There Evidence for a Hubble Bubble? The Nature of Type Ia Supernova Colors and Dust in External Galaxies. The Astrophysical Journal, 664(1), L13. https://doi.org/10.1086/520625

Conley, A., Guy, J., Sullivan, M., Regnault, N., Astier, P., Balland, C., Basa, S., Carlberg, R., Dominique, F., Hardin, D., Hook, I., Howell, D., Pain, R., Palanque-Delabrouille, N., Perrett, K., Pritchet, C., Rich, J., Ruhlmann-Kleider, V., Balam, D., & Walker, and. (2010). Supernova Constraints and Systematic Uncertainties from the First Three Years of the Supernova Legacy Survey. The Astrophysical Journal Supplement Series, 192, 1. https://doi.org/10.1088/0067-0049/192/1/1

Cooray, A. (2002). Integrated Sachs-Wolfe effect: Large scale structure correlation. Physical Review D, 65(10), 103510. https://doi.org/10.1103/PhysRevD.65.103510

Corasaniti, P.-S., Giannantonio, T., & Melchiorri, A. (2005). Constraining dark energy with cross-correlated CMB and large scale structure data. Physical Review D, 71(12), 123521. https://doi.org/10.1103/PhysRevD.71.123521

Core condensation in heavy halos: A two-stage theory for galaxy formation and clustering. — NASA/ADS. (n.d.). Retrieved April 7, 2024, from https://ui.adsabs.harvard.edu/abs/1978MNRAS.183..341W/abstract

Cosmic microwave background theory | PNAS. (n.d.). Retrieved April 6, 2024, from https://www.pnas.org/doi/10.1073/pnas.95.1.35

Cyr-Racine, F.-Y., de Putter, R., Raccanelli, A., & Sigurdson, K. (2014). Constraints on large-scale dark acoustic oscillations from cosmology. Physical Review D, 89(6), 063517. https://doi.org/10.1103/PhysRevD.89.063517

Das, U., & Mukhopadhyay, B. (2013). New Mass Limit for White Dwarfs: Super-Chandrasekhar Type Ia Supernova as a New Standard Candle. Physical Review Letters, 110(7), 071102. https://doi.org/10.1103/PhysRevLett.110.071102

Davis, M., Summers, F. J., & Schlegel, D. (1992). Large-scale structure in a universe with mixed hot and cold dark matter. Nature, 359(6394), 393–396. https://doi.org/10.1038/359393a0

Davis, T. M., & Parkinson, D. (2016). Characterizing Dark Energy Through Supernovae. In A. W. Alsabti & P. Murdin (Eds.), Handbook of Supernovae (pp. 1–23). Springer International Publishing. https://doi.org/10.1007/978-3-319-20794-0_106-1

Dekel, A., Stoehr, F., Mamon, G. A., Cox, T. J., Novak, G. S., & Primack, J. R. (2005). Lost & Found Dark Matter in Elliptical Galaxies. Nature, 437(7059), 707–710. https://doi.org/10.1038/nature03970

Delabrouille, J., Bernardis, P. de, Bouchet, F. R., Achúcarro, A., Ade, P. A. R., Allison, R., Arroja, F., Artal, E., Ashdown, M., Baccigalupi, C., Ballardini, M., Banday, A. J., Banerji, R., Barbosa, D., Bartlett, J., Bartolo, N., Basak, S., Baselmans, J. J. A., Basu, K., … Zannoni, M. (2018). Exploring cosmic origins with CORE: Survey requirements and mission design. Journal of Cosmology and Astroparticle Physics, 2018(04), 014. https://doi.org/10.1088/1475-7516/2018/04/014

Dine, M. (2004). Dark matter and dark energy: A physicist’s perspective. In M. Livio (Ed.), The Dark Universe: Matter, Energy and Gravity (pp. 183–193). Cambridge University Press. https://doi.org/10.1017/CBO9780511536298.016

Double-Disk Dark Matter — ScienceDirect. (n.d.). Retrieved April 12, 2024, from https://www.sciencedirect.com/science/article/pii/S2212686413000289?via%3Dihub

Dvorkin, C., Blum, K., & Kamionkowski, M. (2014). Constraining dark matter-baryon scattering with linear cosmology. Physical Review D, 89(2), 023519. https://doi.org/10.1103/PhysRevD.89.023519

Essig, R., Mardon, J., & Volansky, T. (2012). Direct detection of sub-GeV dark matter. Physical Review D, 85(7), 076007. https://doi.org/10.1103/PhysRevD.85.076007

Evidence of dark energy in different cosmological observations | The European Physical Journal Special Topics. (n.d.). Retrieved April 12, 2024, from https://link-springer-com.ezproxy2.library.colostate.edu/article/10.1140/epjs/s11734-021-00212-y

Evolution of matter and galaxy clustering in cosmological hydrodynamical simulations | Monthly Notices of the Royal Astronomical Society | Oxford Academic. (n.d.). Retrieved April 12, 2024, from https://academic.oup.com/mnras/article/523/3/4693/7192437

Fedderke, M. A., Graham, P. W., & Rajendran, S. (2019). Axion dark matter detection with CMB polarization. Physical Review D, 100(1), 015040. https://doi.org/10.1103/PhysRevD.100.015040

Ferreras, I. (2019). The growth of density fluctuations. In Fundamentals of Galaxy Dynamics, Formation and Evolution (pp. 121–148). UCL Press. https://doi.org/10.2307/j.ctv8jnzhq.13

Folatelli, G., Phillips, M. M., Burns, C. R., Contreras, C., Hamuy, M., Freedman, W. L., Persson, S. E., Stritzinger, M., Suntzeff, N. B., Krisciunas, K., Boldt, L., González, S., Krzeminski, W., Morrell, N., Roth, M., Salgado, F., Madore, B. F., Murphy, D., Wyatt, P., … Miller, N. (2009). THE CARNEGIE SUPERNOVA PROJECT: ANALYSIS OF THE FIRST SAMPLE OF LOW-REDSHIFT TYPE-Ia SUPERNOVAE*. The Astronomical Journal, 139(1), 120. https://doi.org/10.1088/0004-6256/139/1/120

Freeman, K. C. (1987). Dark Matter in Dwarf Galaxies. In S. M. Faber (Ed.), Nearly Normal Galaxies (pp. 317–325). Springer. https://doi.org/10.1007/978-1-4612-4762-3_37

Frieman, J., Turner, M., & Huterer, D. (2008). Dark Energy and the Accelerating Universe. Annual Review of Astronomy and Astrophysics, 46(1), 385–432. https://doi.org/10.1146/annurev.astro.46.060407.145243

Fukugita, M., & Kawasaki, M. (1994). Reionization during hierarchical clustering in a universe dominated by cold dark matter. Monthly Notices of the Royal Astronomical Society, 269(3), 563–578. https://doi.org/10.1093/mnras/269.3.563

Geach, J. E., Peacock, J. A., Myers, A. D., Hickox, R. C., Burchard, M. C., & Jones, M. L. (2019). The Halo Mass of Optically Luminous Quasars at z ≈ 1–2 Measured via Gravitational Deflection of the Cosmic Microwave Background. The Astrophysical Journal, 874(1), 85. https://doi.org/10.3847/1538-4357/ab0894

Geng, C.-Q., Lee, C.-C., & Saridakis, E. N. (2012). Observational constraints on teleparallel dark energy. Journal of Cosmology and Astroparticle Physics, 2012(01), 002. https://doi.org/10.1088/1475-7516/2012/01/002

Gerhard, O. (2010). Pattern speeds in the Milky Way. Memorie Della Societa Astronomica Italiana — Journal of the Italian Astronomical Society, 18.

Gnedin, O. Y., & Ostriker, J. P. (2001). Limits on Collisional Dark Matter from Elliptical Galaxies in Clusters. The Astrophysical Journal, 561(1), 61. https://doi.org/10.1086/323211

Goliath, M., Amanullah, R., Astier, P., Goobar, A., & Pain, R. (2001). Supernovae and the nature of the dark energy. Astronomy & Astrophysics, 380(1), Article 1. https://doi.org/10.1051/0004-6361:20011398

Graham, P. W., Mardon, J., & Rajendran, S. (2016). Vector dark matter from inflationary fluctuations. Physical Review D, 93(10), 103520. https://doi.org/10.1103/PhysRevD.93.103520

Grande, J., Opher, R., Pelinson, A., & Sola, J. (2007). Effective growth of matter density fluctuations in the running LCDM and LXCDM models. Journal of Cosmology and Astroparticle Physics, 2007(12), 007–007. https://doi.org/10.1088/1475-7516/2007/12/007

Guo, Q., Hu, H., Zheng, Z., Liao, S., Du, W., Mao, S., Jiang, L., Wang, J., Peng, Y., Gao, L., Wang, J., & Wu, H. (2020). Further evidence for a population of dark-matter-deficient dwarf galaxies. Nature Astronomy, 4(3), 246–251. https://doi.org/10.1038/s41550-019-0930-9

Guth, A. H., & Pi, S.-Y. (1982). Fluctuations in the New Inflationary Universe. Physical Review Letters, 49(15), 1110–1113. https://doi.org/10.1103/PhysRevLett.49.1110

Harvey, D., Massey, R., Kitching, T., Taylor, A., & Tittley, E. (2015). The nongravitational interactions of dark matter in colliding galaxy clusters. Science, 347(6229). https://doi.org/10.1126/science.1261381

Hu, W. (1998). Structure Formation with Generalized Dark Matter. The Astrophysical Journal, 506(2), 485. https://doi.org/10.1086/306274

Hu, W. (2001). Dark Synergy: Gravitational Lensing and the CMB. Physical Review D, 65(2), 023003. https://doi.org/10.1103/PhysRevD.65.023003

Hu, W. (2003). CMB Temperature and Polarization Anisotropy Fundamentals. Annals of Physics, 303(1), 203–225. https://doi.org/10.1016/S0003-4916(02)00022-2

Huang, Q.-G., & Gong, Y. (2004). Supernova constraints on a holographic dark energy model. Journal of Cosmology and Astroparticle Physics, 2004(08), 006. https://doi.org/10.1088/1475-7516/2004/08/006

Huterer, D., Kirkby, D., Bean, R., Connolly, A., Dawson, K., Dodelson, S., Evrard, A., Jain, B., Jarvis, M., Linder, E., Mandelbaum, R., May, M., Raccanelli, A., Reid, B., Rozo, E., Schmidt, F., Sehgal, N., Slosar, A., van Engelen, A., … Zhao, G. (2015). Growth of cosmic structure: Probing dark energy beyond expansion. Astroparticle Physics, 63, 23–41. https://doi.org/10.1016/j.astropartphys.2014.07.004

Huterer, D., & Shafer, D. L. (2017). Dark energy two decades after: Observables, probes, consistency tests. Reports on Progress in Physics, 81(1), 016901. https://doi.org/10.1088/1361-6633/aa997e

Improving cosmological parameter estimation with the future 21 cm observation from SKA — ScienceDirect. (n.d.). Retrieved April 12, 2024, from https://www.sciencedirect.com/science/article/pii/S0370269319307865?via%3Dihub

Kahniashvili, T., & Ratra, B. (2007). CMB anisotropies due to cosmological magnetosonic waves. Physical Review D, 75(2), 023002. https://doi.org/10.1103/PhysRevD.75.023002

Kamada, A., Kaplinghat, M., Pace, A. B., & Yu, H.-B. (2017). How the Self-Interacting Dark Matter Model Explains the Diverse Galactic Rotation Curves. Physical Review Letters, 119(11), 111102. https://doi.org/10.1103/PhysRevLett.119.111102

Kelso, C., Profumo, S., & Queiroz, F. S. (2013). Nonthermal WIMPs as ``dark radiation’’ in light of ATACAMA, SPT, WMAP9, and Planck. Physical Review D, 88(2), 023511. https://doi.org/10.1103/PhysRevD.88.023511

Klypin, A., Poulin, V., Prada, F., Primack, J., Kamionkowski, M., Avila-Reese, V., Rodriguez-Puebla, A., Behroozi, P., Hellinger, D., & Smith, T. L. (2021). Clustering and Halo Abundances in Early Dark Energy Cosmological Models. Monthly Notices of the Royal Astronomical Society, 504(1), 769–781. https://doi.org/10.1093/mnras/stab769

Kneller, J. P., & Steigman, G. (2003). Big bang nucleosynthesis and CMB constraints on dark energy. Physical Review D, 67(6), 063501. https://doi.org/10.1103/PhysRevD.67.063501

Komatsu, E., Bennett, C. L., (on behalf of the WMAP science team), Barnes, C., Bean, R., Bennett, C. L., Dore, O., Dunkley, J., Gold, B., Greason, M. R., Halpern, M., Hill, R. S., Hinshaw, G., Jarosik, N., Kogut, A., Komatsu, E., Larson, D., Limon, M., Meyer, S. S., … Wright, E. L. (2014). Results from the Wilkinson Microwave Anisotropy Probe. Progress of Theoretical and Experimental Physics, 2014(6), 6B102–0. https://doi.org/10.1093/ptep/ptu083

Komatsu, E., Smith, K. M., Dunkley, J., Bennett, C. L., Gold, B., Hinshaw, G., Jarosik, N., Larson, D., Nolta, M. R., Page, L., Spergel, D. N., Halpern, M., Hill, R. S., Kogut, A., Limon, M., Meyer, S. S., Odegard, N., Tucker, G. S., Weiland, J. L., … Wright, E. L. (2011). Seven-year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Interpretation. The Astrophysical Journal Supplement Series, 192, 18. https://doi.org/10.1088/0067-0049/192/2/18

Krall, R., Cyr-Racine, F.-Y., & Dvorkin, C. (2017). Wandering in the Lyman-alpha forest: A study of dark matter-dark radiation interactions. Journal of Cosmology and Astroparticle Physics, 2017(09), 003. https://doi.org/10.1088/1475-7516/2017/09/003

Krolewski, A., & Ferraro, S. (2022). The Integrated Sachs Wolfe effect: unWISE and Planck constraints on dynamical dark energy. Journal of Cosmology and Astroparticle Physics, 2022(04), 033. https://doi.org/10.1088/1475-7516/2022/04/033

Lahav, O. (2020). Dark Energy: Is it `just’ Einstein’s Cosmological Constant Lambda? Contemporary Physics, 61(2), 132–145. https://doi.org/10.1080/00107514.2020.1837456

Laureijs, R., Gondoin, P., Duvet, L., Criado, G. S., Hoar, J., Amiaux, J., Auguères, J.-L., Cole, R., Cropper, M., Ealet, A., Ferruit, P., Sanz, I. E., Jahnke, K., Kohley, R., Maciaszek, T., Mellier, Y., Oosterbroek, T., Pasian, F., Sauvage, M., … Valenziano, L. (2012). Euclid: ESA’s mission to map the geometry of the dark universe. Space Telescopes and Instrumentation 2012: Optical, Infrared, and Millimeter Wave, 8442, 329–336. https://doi.org/10.1117/12.926496

Lee, H. S. (2019). Direct Detection of Dark Matter. In Proceedings of The 39th International Conference on High Energy Physics — PoS(ICHEP2018) (Vol. 340, p. 728). SISSA Medialab. https://doi.org/10.22323/1.340.0728

Lewis, A., & Challinor, A. (2006). Weak gravitational lensing of the CMB. Physics Reports, 429(1), 1–65. https://doi.org/10.1016/j.physrep.2006.03.002

Li, B., Barrow, J. D., Mota, D. F., & Zhao, H. (2008). Testing alternative theories of dark matter with the CMB. Physical Review D, 78(6), 064021. https://doi.org/10.1103/PhysRevD.78.064021

Liddle, A. R., & Lyth, D. H. (1993). The Cold Dark Matter Density Perturbation. Physics Reports, 231(1–2), 1–105. https://doi.org/10.1016/0370-1573(93)90114-S

Loeb, A., & Weiner, N. (2011). Cores in Dwarf Galaxies from Dark Matter with a Yukawa Potential. Physical Review Letters, 106(17), 171302. https://doi.org/10.1103/PhysRevLett.106.171302

Longair, M. S., & Smeenk, C. (2019). Inflation, dark matter, and dark energy. In H. Kragh & M. S. Longair (Eds.), The Oxford Handbook of the History of Modern Cosmology (p. 0). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198817666.013.11

Ludwig, G. O. (2021). Galactic rotation curve and dark matter according to gravitomagnetism. The European Physical Journal C, 81(2), 186. https://doi.org/10.1140/epjc/s10052-021-08967-3

Meerburg, D. (2021). Squeezing down the Theory Space for Cosmic Inflation. Physics, 14, 135. https://doi.org/10.1103/Physics.14.135

Mellier, Y. (2010). Gravitational lensing and dark matter. In G. Bertone (Ed.), Particle Dark Matter: Observations, Models and Searches (pp. 56–82). Cambridge University Press. https://doi.org/10.1017/CBO9780511770739.005

Miyatake, H., Harikane, Y., Ouchi, M., Ono, Y., Yamamoto, N., Nishizawa, A. J., Bahcall, N., Miyazaki, S., & Malagón, A. A. P. (2022). First Identification of a CMB Lensing Signal Produced by 1.5 Million Galaxies: Constraints on Matter Density Fluctuations at High Redshift. Physical Review Letters, 129(6), 061301. https://doi.org/10.1103/PhysRevLett.129.061301

Mo, H. J., & Mao, S. (2000). The Tully-Fisher relation and its implications for the halo density profile and self-interacting dark matter. Monthly Notices of the Royal Astronomical Society, 318(1), 163–172. https://doi.org/10.1046/j.1365-8711.2000.03714.x

Mo, H., van den Bosch, F. C., & White, S. (2010). Galaxy Formation and Evolution. In Galaxy Formation and Evolution. https://ui.adsabs.harvard.edu/abs/2010gfe..book.....M

Mörtsell, E., & Dhawan, S. (2018). Does the Hubble constant tension call for new physics? Journal of Cosmology and Astroparticle Physics, 2018(09), 025–025. https://doi.org/10.1088/1475-7516/2018/09/025

Motta, V., García-Aspeitia, M. A., Hernández-Almada, A., Magaña, J., & Verdugo, T. (2021). Taxonomy of Dark Energy Models (arXiv:2104.04642). arXiv. https://doi.org/10.48550/arXiv.2104.04642

Mould, J., Han, M., & Bothun, G. (1989). Nonlinearity of the Tully-Fisher Relation. The Astrophysical Journal, 347, 112. https://doi.org/10.1086/168101

Murgia, R., Merle, A., Viel, M., Totzauer, M., & Schneider, A. (2017). “Non-cold” dark matter at small scales: A general approach. Journal of Cosmology and Astroparticle Physics, 2017(11), 046. https://doi.org/10.1088/1475-7516/2017/11/046

Naoz, S., & Barkana, R. (2005). Growth of linear perturbations before the era of the first galaxies. Monthly Notices of the Royal Astronomical Society, 362(3), 1047–1053. https://doi.org/10.1111/j.1365-2966.2005.09385.x

Năstase, H. (2019). Evidence for Dark Matter and the Lambda CDM Model. In H. Năstase (Ed.), Cosmology and String Theory (pp. 41–51). Springer International Publishing. https://doi.org/10.1007/978-3-030-15077-8_4

Natarajan, P. (1999). Evidence for dark matter in clusters from lensing studies. AIP Conference Proceedings, 478(1), 295–298. https://doi.org/10.1063/1.59406

Navarro, J. F., Frenk, C. S., & White, S. D. M. (1995). The assembly of galaxies in a hierarchically clustering universe. Monthly Notices of the Royal Astronomical Society, 275(1), 56–66. https://doi.org/10.1093/mnras/275.1.56

Newburgh, L. B. (2014). The Cosmic Microwave Background: An Experimentalists&#39;s Guide to CMB Measurements and Prospects for the Future. In Proceedings of Frank N. Bash Symposium 2013: New Horizons in Astronomy — PoS(BASH 2013) (Vol. 206, p. 001). SISSA Medialab. https://doi.org/10.22323/1.206.0001

O’Raifeartaigh, C., & McCann, B. (2014). Einstein’s cosmic model of 1931 revisited: An analysis and translation of a forgotten model of the universe. The European Physical Journal H, 39(1), 63–85. https://doi.org/10.1140/epjh/e2013-40038-x

Padmanabhan, N., & Finkbeiner, D. P. (2005). Detecting Dark Matter Annihilation with CMB Polarization: Signatures and Experimental Prospects. Physical Review D, 72(2), 023508. https://doi.org/10.1103/PhysRevD.72.023508

Parker, B. (1989). Gravitational Lenses and Dark Matter. In B. Parker (Ed.), Invisible Matter and the Fate of the Universe (pp. 183–200). Springer US. https://doi.org/10.1007/978-1-4899-6469-4_11

Peacock, J. A., Cole, S., Norberg, P., Baugh, C. M., Bland-Hawthorn, J., Bridges, T., Cannon, R. D., Colless, M., Collins, C., Couch, W., Dalton, G., Deeley, K., De Propris, R., Driver, S. P., Efstathiou, G., Ellis, R. S., Frenk, C. S., Glazebrook, K., Jackson, C., … Taylor, K. (2001). A measurement of the cosmological mass density from clustering in the 2dF Galaxy Redshift Survey. Nature, 410(6825), 169–173. https://doi.org/10.1038/35065528

Perlmutter, S., Aldering, G., Goldhaber, G., Knop, R. A., Nugent, P., Castro, P. G., Deustua, S., Fabbro, S., Goobar, A., Groom, D. E., Hook, I. M., Kim, A. G., Kim, M. Y., Lee, J. C., Nunes, N. J., Pain, R., Pennypacker, C. R., Quimby, R., Lidman, C., … Project, T. S. C. (1999). Measurements of Ω and Λ from 42 High-Redshift Supernovae. The Astrophysical Journal, 517(2), 565. https://doi.org/10.1086/307221

Perlmutter, S., Turner, M. S., & White, M. (1999). Constraining Dark Energy with Type Ia Supernovae and Large-Scale Structure. Physical Review Letters, 83(4), 670–673. https://doi.org/10.1103/PhysRevLett.83.670

Persic, M., Salucci, P., & Stel, F. (1999). The Universal Rotation Curve of Spiral Galaxies: I. the Dark Matter Connection. https://doi.org/10.1093/mnras/281.1.27

Pettorino, V., Amendola, L., Baccigalupi, C., & Quercellini, C. (2012). Constraints on coupled dark energy using CMB data from WMAP and South Pole Telescope. Physical Review D, 86(10), 103507. https://doi.org/10.1103/PhysRevD.86.103507

Phys. Rev. D 65, 103510 (2002) — Integrated Sachs-Wolfe effect: Large scale structure correlation. (n.d.). Retrieved April 11, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.65.103510

Phys. Rev. D 74, 043505 (2006) — Can a galaxy redshift survey measure dark energy clustering? (n.d.). Retrieved April 12, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.74.043505

Phys. Rev. D 77, 103507 (2008) — Testing cosmology with cosmic sound waves. (n.d.). Retrieved April 12, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.77.103507

Phys. Rev. D 82, 023504 (2010) — Constraints on cold dark matter accelerating cosmologies and cluster formation. (n.d.). Retrieved April 12, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.82.023504

Phys. Rev. D 82, 083520 (2010) — Relative velocity of dark matter and baryonic fluids and the formation of the first structures. (n.d.). Retrieved April 12, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.82.083520

Phys. Rev. D 85, 076007 (2012) — Direct detection of sub-GeV dark matter. (n.d.). Retrieved April 12, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.85.076007

Phys. Rev. D 101, 023512 (2020) — Ray tracing the integrated Sachs-Wolfe effect through the light cones of the dark energy universe simulation-full universe runs. (n.d.). Retrieved April 11, 2024, from https://journals.aps.org/prd/abstract/10.1103/PhysRevD.101.023512

Planck 2015 results — XXI. The integrated Sachs-Wolfe effect | Astronomy & Astrophysics (A&A). (n.d.). Retrieved April 11, 2024, from https://www.aanda.org/articles/aa/full_html/2016/10/aa25831-15/aa25831-15.html

Planck captures portrait of the young Universe, revealing earliest light. (2013, March 21). University of Cambridge. https://www.cam.ac.uk/research/news/planck-captures-portrait-of-the-young-universe-revealing-earliest-light

Planck Collaboration, Ade, P. A. R., Aghanim, N., Armitage-Caplan, C., Arnaud, M., Ashdown, M., Atrio-Barandela, F., Aumont, J., Baccigalupi, C., Banday, A. J., Barreiro, R. B., Bartlett, J. G., Battaner, E., Benabed, K., Benoît, A., Benoit-Lévy, A., Bernard, J.-P., Bersanelli, M., Bielewicz, P., … Zonca, A. (2014). Planck 2013 results. XVI. Cosmological parameters. Astronomy & Astrophysics, 571, A16. https://doi.org/10.1051/0004-6361/201321591

Planck Collaboration, Aghanim, N., Akrami, Y., Alves, M. I. R., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., … Zonca, A. (2020). Planck 2018 results. XII. Galactic astrophysics using polarized dust emission. Astronomy & Astrophysics, 641, A12. https://doi.org/10.1051/0004-6361/201833885

Planck Collaboration, Aghanim, N., Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Battye, R., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., … Zonca, A. (2021). Planck 2018 results. VI. Cosmological parameters. https://doi.org/10.1051/0004-6361/201833910

Planck Collaboration, Aghanim, N., Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., Bouchet, F. R., … Zonca, A. (2020a). Planck 2018 results. V. CMB power spectra and likelihoods. Astronomy & Astrophysics, 641, A5. https://doi.org/10.1051/0004-6361/201936386

Planck Collaboration, Aghanim, N., Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., Bouchet, F. R., … Zonca, A. (2020b). Planck 2018 results. VIII. Gravitational lensing. Astronomy & Astrophysics, 641, A8. https://doi.org/10.1051/0004-6361/201833886

Planck Collaboration, Aghanim, N., Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bond, J. R., Borrill, J., Bouchet, F. R., Boulanger, F., … Zonca, A. (2020c). Planck 2018 results. III. High Frequency Instrument data processing and frequency maps. Astronomy & Astrophysics, 641, A3. https://doi.org/10.1051/0004-6361/201832909

Planck Collaboration, Akrami, Y., Argüeso, F., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bonavera, L., Bond, J. R., Borrill, J., Bouchet, F. R., … Zonca, A. (2020a). Planck 2018 results. II. Low Frequency Instrument data processing. Astronomy & Astrophysics, 641, A2. https://doi.org/10.1051/0004-6361/201833293

Planck Collaboration, Akrami, Y., Arroja, F., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Battye, R., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., … Zonca, A. (2020b). Planck 2018 results. I. Overview and the cosmological legacy of Planck. Astronomy & Astrophysics, 641, A1. https://doi.org/10.1051/0004-6361/201833880

Planck Collaboration, Akrami, Y., Arroja, F., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., Bouchet, F. R., … Zonca, A. (2020c). Planck 2018 results. X. Constraints on inflation. Astronomy & Astrophysics, 641, A10. https://doi.org/10.1051/0004-6361/201833887

Planck Collaboration, Akrami, Y., Arroja, F., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bond, J. R., Borrill, J., Bouchet, F. R., Bucher, M., … Zonca, A. (2019). Planck 2018 results. IX. Constraints on primordial non-Gaussianity (arXiv:1905.05697). arXiv. https://doi.org/10.48550/arXiv.1905.05697

Planck Collaboration, Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bernard, J.-P., Bersanelli, M., Bielewicz, P., Bond, J. R., Borrill, J., Bouchet, F. R., Boulanger, F., Bracco, A., … Zonca, A. (2020d). Planck 2018 results. XI. Polarized dust foregrounds. Astronomy & Astrophysics, 641, A11. https://doi.org/10.1051/0004-6361/201832618

Planck Collaboration, Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bersanelli, M., Bielewicz, P., Bock, J. J., Bond, J. R., Borrill, J., Bouchet, F. R., Boulanger, F., Bucher, M., … Zonca, A. (2020e). Planck 2018 results. VII. Isotropy and Statistics of the CMB. Astronomy & Astrophysics, 641, A7. https://doi.org/10.1051/0004-6361/201935201

Planck Collaboration, Akrami, Y., Ashdown, M., Aumont, J., Baccigalupi, C., Ballardini, M., Banday, A. J., Barreiro, R. B., Bartolo, N., Basak, S., Benabed, K., Bersanelli, M., Bielewicz, P., Bond, J. R., Borrill, J., Bouchet, F. R., Boulanger, F., Bucher, M., Burigana, C., … Zonca, A. (2020f). Planck 2018 results. IV. Diffuse component separation. Astronomy & Astrophysics, 641, A4. https://doi.org/10.1051/0004-6361/201833881

Pogosian, L., Corasaniti, P. S., Stephan-Otto, C., Crittenden, R., & Nichol, R. (2005). Tracking dark energy with the integrated Sachs-Wolfe effect: Short and long-term predictions. Physical Review D, 72(10), 103519. https://doi.org/10.1103/PhysRevD.72.103519

Popovic, B., Brout, D., Kessler, R., & Scolnic, D. (2023). The Pantheon+ Analysis: Forward Modeling the Dust and Intrinsic Color Distributions of Type Ia Supernovae, and Quantifying Their Impact on Cosmological Inferences. The Astrophysical Journal, 945(1), 84. https://doi.org/10.3847/1538-4357/aca273

Probing decisive answers to dark energy questions from cosmic complementarity and lensing tomography | Monthly Notices of the Royal Astronomical Society | Oxford Academic. (n.d.). Retrieved April 12, 2024, from https://academic.oup.com/mnras/article/363/2/469/1125060

Rauch, M. (1998). THE LYMAN ALPHA FOREST IN THE SPECTRA OF QUASISTELLAR OBJECTS. Annual Review of Astronomy and Astrophysics, 36(Volume 36, 1998), 267–316. https://doi.org/10.1146/annurev.astro.36.1.267

Riess, A. G. (2000). The Case for an Accelerating Universe from Supernovae. Publications of the Astronomical Society of the Pacific, 112(776), 1284. https://doi.org/10.1086/316624

Riess, A. G., Strolger, L.-G., Casertano, S., Ferguson, H. C., Mobasher, B., Gold, B., Challis, P. J., Filippenko, A. V., Jha, S., Li, W., Tonry, J., Foley, R., Kirshner, R. P., Dickinson, M., MacDonald, E., Eisenstein, D., Livio, M., Younger, J., Xu, C., … Stern, D. (2007). New Hubble Space Telescope Discoveries of Type Ia Supernovae at z ≥ 1: Narrowing Constraints on the Early Behavior of Dark Energy*. The Astrophysical Journal, 659(1), 98. https://doi.org/10.1086/510378

Riess, A. G., Strolger, L.-G., Tonry, J., Casertano, S., Ferguson, H. C., Mobasher, B., Challis, P., Filippenko, A. V., Jha, S., Li, W., Chornock, R., Kirshner, R. P., Leibundgut, B., Dickinson, M., Livio, M., Giavalisco, M., Steidel, C. C., Benítez, T., & Tsvetanov, Z. (2004). Type Ia Supernova Discoveries at z > 1 from the Hubble Space Telescope: Evidence for Past Deceleration and Constraints on Dark Energy Evolution*. The Astrophysical Journal, 607(2), 665. https://doi.org/10.1086/383612

Robson, B. A. (2019). Introductory Chapter: Standard Model of Cosmology. In Redefining Standard Model Cosmology. IntechOpen. https://doi.org/10.5772/intechopen.85605

Roos, M. (2008). Expansion of the Universe — Standard Big Bang Model (arXiv:0802.2005). arXiv. https://doi.org/10.48550/arXiv.0802.2005

Rubin, D., & Heitlauf, J. (2020). Is the Expansion of the Universe Accelerating? All Signs Still Point to Yes: A Local Dipole Anisotropy Cannot Explain Dark Energy. The Astrophysical Journal, 894(1), 68. https://doi.org/10.3847/1538-4357/ab7a16

Rubin, V. C. (1993). Galaxy dynamics and the mass density of the universe. Proceedings of the National Academy of Sciences of the United States of America, 90(11), 4814–4821.

Rubin, V. C., Thonnard, N., & Ford, W. K., Jr. (1980). Rotational properties of 21 SC galaxies with a large range of luminosities and radii, from NGC 4605 /R = 4kpc/ to UGC 2885 /R = 122 kpc/. The Astrophysical Journal, 238, 471. https://doi.org/10.1086/158003

Sahni, V. (2003). Theoretical models of dark energy. Chaos, Solitons & Fractals, 16(4), 527–537. https://doi.org/10.1016/S0960-0779(02)00221-7

Schimd, C. (2004). Weak Lensing in Scalar-Tensor Theories of Gravity: Preliminary Results. Proceedings of the International Astronomical Union, 2004(IAUS225), 129–139. https://doi.org/10.1017/S1743921305001900

Schmidt, B. P. (2003). Evidence from type Ia supernovae for an accelerating Universe. Chaos, Solitons & Fractals, 16(4), 479–492. https://doi.org/10.1016/S0960-0779(02)00217-5

Sheldon, E. S., Becker, M. R., Jarvis, M., Armstrong, R., & Collaboration, L. D. E. S. (2023). Metadetection Weak Lensing for the Vera C. Rubin Observatory. The Open Journal of Astrophysics, 6. https://doi.org/10.21105/astro.2303.03947

Signatures of self-interacting dark matter in the matter power spectrum and the CMB — ScienceDirect. (n.d.). Retrieved April 11, 2024, from https://www.sciencedirect.com/science/article/pii/S0370269318304726?via%3Dihub

Silk, J. (2016). Challenges in Cosmology from the Big Bang to Dark Energy, Dark Matter and Galaxy Formation.

Simulating the physical properties of dark matter and gas inside the cosmic web | Monthly Notices of the Royal Astronomical Society | Oxford Academic. (n.d.). Retrieved April 12, 2024, from https://academic.oup.com/mnras/article/370/2/656/967477

Slatyer, T. R. (2016). Indirect dark matter signatures in the cosmic dark ages. I. Generalizing the bound on $s$-wave dark matter annihilation from Planck results. Physical Review D, 93(2), 023527. https://doi.org/10.1103/PhysRevD.93.023527

Sofue, Y., & Rubin, V. (2001). Rotation Curves of Spiral Galaxies. Annual Review of Astronomy and Astrophysics, 39(1), 137–174. https://doi.org/10.1146/annurev.astro.39.1.137

Spergel, D. N., Bean, R., Doré, O., Nolta, M. R., Bennett, C. L., Dunkley, J., Hinshaw, G., Jarosik, N., Komatsu, E., Page, L., Peiris, H. V., Verde, L., Halpern, M., Hill, R. S., Kogut, A., Limon, M., Meyer, S. S., Odegard, N., Tucker, G. S., … Wright, E. L. (2007). Wilkinson Microwave Anisotropy Probe (WMAP) Three Year Results: Implications for Cosmology. The Astrophysical Journal Supplement Series, 170(2), 377–408. https://doi.org/10.1086/513700

Springel, V., White, S. D. M., Jenkins, A., Frenk, C. S., Yoshida, N., Gao, L., Navarro, J., Thacker, R., Croton, D., Helly, J., Peacock, J. A., Cole, S., Thomas, P., Couchman, H., Evrard, A., Colberg, J., & Pearce, F. (2005). Simulations of the formation, evolution and clustering of galaxies and quasars. Nature, 435(7042), 629–636. https://doi.org/10.1038/nature03597

Staggs, S. T. (2003). Repercussions of Structure Emergence on the CMB Polarization. AIP Conference Proceedings, 666(1), 59–66. https://doi.org/10.1063/1.1581771

Standard cosmological model. Big Bang. Forming the structure of the universe. (n.d.). Retrieved April 6, 2024, from https://astronuclphysics.info/Gravitace5-4.htm

Szydlowski, M., & Tambor, P. (2008). Emergence and Effective Theory of the Universe — The Case Study of Lambda Cold Dark Matter Cosmological Model.

Taylor, A. N., & Rowan-Robinson, M. (1992). The spectrum of cosmological density fluctuations and nature of dark matter. Nature, 359(6394), 396–399. https://doi.org/10.1038/359396a0

Technology, S. U. of. (n.d.). Dark energy discovery a decade in the making: New supernova insights offer clues to the expansion of the universe. Retrieved April 12, 2024, from https://phys.org/news/2024-01-dark-energy-discovery-decade-supernova.html

Tenkanen, T. (2019). Dark Matter from Scalar Field Fluctuations. Physical Review Letters, 123(6), 061302. https://doi.org/10.1103/PhysRevLett.123.061302

The Early Universe and Observational Cosmology | SpringerLink. (n.d.). Retrieved April 11, 2024, from https://link-springer-com.ezproxy2.library.colostate.edu/book/10.1007/b97189

Three-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Implications for Cosmology — NASA/ADS. (n.d.). Retrieved April 7, 2024, from https://ui.adsabs.harvard.edu/abs/2007ApJS..170..377S/abstract

Tsujikawa, S., Uddin, K., Mizuno, S., Tavakol, R., & Yokoyama, J. (2008). Constraints on scalar-tensor models of dark energy from observational and local gravity tests. Physical Review D, 77(10), 103009. https://doi.org/10.1103/PhysRevD.77.103009

Tully, R. B., & Fisher, J. R. (1977). A new method of determining distances to galaxies. Astronomy and Astrophysics, 54, 661–673.

Type Ia Supernovae and Cosmology | SpringerLink. (n.d.). Retrieved April 11, 2024, from https://link-springer-com.ezproxy2.library.colostate.edu/chapter/10.1007/978-3-642-10598-2_2

Undagoitia, T. M., & Rauch, L. (2015). Dark matter direct-detection experiments. Journal of Physics G: Nuclear and Particle Physics, 43(1), 013001. https://doi.org/10.1088/0954-3899/43/1/013001

Very Large Scale Structure in an Open Cosmology of Cold Dark Matter and Baryons — NASA/ADS. (n.d.). Retrieved April 6, 2024, from https://ui.adsabs.harvard.edu/abs/1988ApJ...326..539B/abstract

Villano, A. N., Harris, K. C., Bergfalk, J., Hatami, R., Vititoe, F., & Johnston, J. (2023a). The Data Behind Dark Matter: Exploring GalacticRotation. Journal of Open Source Education, 6(66), 184. https://doi.org/10.21105/jose.00184

Villano, A. N., Harris, K. C., Bergfalk, J., Hatami, R., Vititoe, F., & Johnston, J. (2023b). The Data Behind Dark Matter: Exploring GalacticRotation. Journal of Open Source Education, 6(66), 184. https://doi.org/10.21105/jose.00184

Vogelsberger, M., Genel, S., Springel, V., Torrey, P., Sijacki, D., Xu, D., Snyder, G., Bird, S., Nelson, D., & Hernquist, L. (2014). Properties of galaxies reproduced by a hydrodynamic simulation. Nature, 509, 177–182. https://doi.org/10.1038/nature13316

Wang, B., Abdalla, E., Atrio-Barandela, F., & Pavón, D. (2024). Further understanding the interaction between dark energy and dark matter: Current status and future directions (arXiv:2402.00819). arXiv. http://arxiv.org/abs/2402.00819

Wang, D. (2022). constraints on dark energy and modified gravity: An evidence of dynamical dark energy. Physical Review D, 106(6), 063515. https://doi.org/10.1103/PhysRevD.106.063515

Wang, Y., & Garnavich, P. M. (2001). Measuring Time Dependence of Dark Energy Density from Type Ia Supernova Data. The Astrophysical Journal, 552(2), 445. https://doi.org/10.1086/320552

Whitbourn, J. R., Shanks, T., & Sawangwit, U. (2014). Testing WMAP data via Planck radio and SZ catalogues. Monthly Notices of the Royal Astronomical Society, 437(1), 622–640. https://doi.org/10.1093/mnras/stt1912

Wright, B. S., & Li, B. (2018). Type Ia supernovae, standardizable candles, and gravity. Physical Review D, 97(8), 083505. https://doi.org/10.1103/PhysRevD.97.083505

YOSHIDA, N. (2019). Formation of the first generation of stars and blackholes in the Universe. Proceedings of the Japan Academy. Series B, Physical and Biological Sciences, 95(1), 17–28. https://doi.org/10.2183/pjab.95.002

Yoshida, N., Sugiyama, N., & Hernquist, L. (2003). The evolution of baryon density fluctuations in multicomponent cosmological simulations. Monthly Notices of the Royal Astronomical Society, 344(2), 481–491. https://doi.org/10.1046/j.1365-8711.2003.06829.x

Zhang, J.-F., Geng, J.-J., & Zhang, X. (2014). Neutrinos and dark energy after Planck and BICEP2: Data consistency tests and cosmological parameter constraints. Journal of Cosmology and Astroparticle Physics, 2014(10), 044. https://doi.org/10.1088/1475-7516/2014/10/044

Zhang, L., Chen, X., Kamionkowski, M., Si, Z., & Zheng, Z. (2007). Constraints on radiative dark-matter decay from the cosmic microwave background. Physical Review D, 76(6), 061301. https://doi.org/10.1103/PhysRevD.76.061301

Zhang, X., & Wu, F.-Q. (2005). Constraints on holographic dark energy from type Ia supernova observations. Physical Review D, 72(4), 043524. https://doi.org/10.1103/PhysRevD.72.043524

Zhao, G.-B., Raveri, M., Pogosian, L., Wang, Y., Crittenden, R. G., Handley, W. J., Percival, W. J., Beutler, F., Brinkmann, J., Chuang, C.-H., Cuesta, A. J., Eisenstein, D. J., Kitaura, F.-S., Koyama, K., L’Huillier, B., Nichol, R. C., Pieri, M. M., Rodriguez-Torres, S., Ross, A. J., … Zhang, H. (2017). Dynamical dark energy in light of the latest observations. Nature Astronomy, 1(9), 627–632. https://doi.org/10.1038/s41550-017-0216-z

Zheng, Y.-J. (2023). A Better Candidate for Dark Matter is Cosmic Plasma. Research in Astronomy and Astrophysics, 23(10), 105007. https://doi.org/10.1088/1674-4527/acf082

Introduction

The innovative camera and high-tech tools at the Vera C. Rubin Observatory have the potential to change how we think about the constantly shifting universe completely. We must be as creative as possible to manage this big project, including complex systems and astronomical data. The observatory design used the well-organized Model-Based Systems Engineering (MBSE) system, but we need a more flexible solution with data-driven answers for telescope scheduling. Artificial intelligence (AI) can change astronomical observation by increasing scheduling efficiency and freedom. When weather trends change, and scientific goals shift, it may be hard to use traditional scheduling methods. However, AI-powered systems can constantly improve plans in real-time (Johnston & Miller, 1989). For example, time-sensitive events like supernovae require the telescope to prepare for the observatory’s total scientific impact (Sebag et al., 2020). This project examines how AI could improve the Rubin Observatory’s scheduling. The goals are to improve scheduling efficiency, increase scientific output, and highlight AI’s transformative prospects within this complex astronomical system (Patil, 2023). In this project, we want to find the best ways to use AI, create a flexible system design, and make a prototype to test the system’s abilities. We want to set a standard for how AI can be used in complex astronomical systems in the future (Solar & Atkinson, 2020). Lastly, for a visual representation of the key concepts discussed in each section, please refer to Appendix A, where images corresponding to the topics are provided. This visual aid helps to enhance understanding and offer a clearer insight into the technological advancements for this report.

Problem Statement

At the Rubin Observatory, the current scheduling methods sometimes leave out important astronomical events that, when they happen immediately, astronomers must spend a lot of time making changes by hand to align the telescope. Astronomers spend several hours in alighted instruments rather than analyzing valuable scientific data, and the capacity to make significant developments is put aside for more mechanical functions (Frank & Kürklü, 2005). For that reason, an AI-enhanced scheduling system can increase the efficiency of recording fleeting events and reduce the time astronomers spend on planning (Patil, 2023). This is done by quickly looking at large, complicated information and adapting to changing conditions. Aside from that, the Rubin Observatory’s long-term scientific effect is limited by its current scheduling issues. Some researchers may have to wait or even give up on promising projects because the observatory isn’t responding to unexpected results, and telescope time isn’t used efficiently. Ultimately, we are looking at how the observatory can use all of its scientific powers and make groundbreaking findings that change how we think about the universe thanks to an AI-enhanced scheduling system, which can get around these challenges (Solar & Atkinson, 2020).

Project Objectives

Our project wants to develop and implement an AI-powered scheduling system that significantly reduces the time astronomers spend on project tasks compared to traditional scheduling methods. Additionally, AI-optimized scheduling aims to enhance the observatory’s ability to find and study important events like supernovae. The main goal is to make a prototype that shows how beneficial AI is in the systems engineering process at the Rubin Observatory. It is possible with AI in complicated areas of science such as astronomy (Alves et al., 2021). Also, we want to enhance the practical efficiency of the observatory’s scheduling processes in addition to its primary scientific goal. The AI system can simplify workflows by eliminating tedious tasks and improving resource sharing. Additionally, we expect an increase in efficiency to free up astronomers’ time and observatory resources, allowing them to focus more intently on cutting-edge research projects and meeting the observatory’s long-term scientific goals (van Rooyen, Maartens, & Martinez, 2018).

Current Applications

AI technologies have been instrumental in improving the powers of astronomical observatories worldwide, greatly enhancing the scheduling and operation of telescopes. These technologies improve the precision of finding heavenly events and the time of observations, which is required for handling the enormous amounts of data modern observatories handle (Gilda et al., 2020). By using real-time weather data to alter monitoring plans dynamically, forecast machine learning algorithms, for instance, are frequently used to guess the best watching conditions. AI expertise is necessary for ensuring that observations are planned under ideal conditions and adapt to sudden changes in the atmosphere (Giordano et al., 2021). Telescopes’ ability to recognize and respond to transient astronomical events, like supernovae or quick radio bursts, is also greatly enhanced by AI, which allows for efficient data processing quickly. By processing data in real-time, these systems can start new observations more seamlessly than before, which maximizes the scientific output from such short-lived events (Neira et al., 2020).

State of AI Augmentation

At prestigious organizations like the European Southern Observatory (ESO) and the Hubble Space Telescope, AI has handled difficult data processing tasks, improved operations, and boosted productivity. These observatories have used AI to improve data processing and automate regular data decisions to reduce human mistakes and operational waste. Based on these improvements, our project at the Rubin Observatory wants to make AI-driven solutions that work better for us regarding scheduling.

It is designed to deal with the Rubin Observatory’s specific problems with its AI-driven scheduling system. It uses complex methods to increase the scheduling of observations in real-time (Dobrzycki et al., 2004). This plan uses trends learned from past data and real-time data from the environment and the stars to look after so that the observatory can quickly adapt to discoveries and changing conditions. Through the development of a prototype, our objective is to demonstrate the real benefits of AI in this situation and to look into its capacity to set new standards for automatic operations in observatories worldwide (Bianco et al., 2021). Incorporating AI into the operational framework of the Rubin Observatory builds on the results of our earlier research. It uses well-known AI methods that have demonstrated their capacity in similar high-risk situations. We want to refine the system efficiency and address more significant problems in systems engineering, such as ensuring data truth, increasing system capacity, and improving the relationship between systems and human intelligence. Our methodology prioritizes studying social issues, checking and validating systems, and collaborating between humans and AI (Guy et al., 2021). By considering these bigger issues, we provide that AI systems are technically advanced, follow the rules of ethics, and work together as needed in the scientific community. Adopting a thorough approach to integrating AI allows the development of solid and ethically sound solutions while also improving the capacity of the Rubin Observatory to make valuable contributions to astronomical research (Lopez et al., 2020).

AI Methods Selection

The AI program at the Rubin Observatory chooses methods to use carefully. This is a very important step because the AI methods must handle the observatory’s massive and always-changing data sets well. Multiple cutting-edge AI methods are being thought about, each with its unique ways to make the observatory work better. In addition to what the AI methods can do in theory, it is very important to think about how they will work in the Rubin Observatory environment, where there are many restrictions and requirements (Johnson et al., 2021). The end choice will be influenced by factors like the amount of computing power available, the need for decision-making in near-real time, and the importance of providing a clear explanation. Finding the method or mix of methods that best balance accuracy, efficiency, and interpretability will require extensive testing and review in a virtual observatory environment (Allam et al., 2023).

Methods Considered

Several promising AI approaches were evaluated. Supervised machine learning algorithms, such as neural networks and decision trees, excel at classification and prediction tasks within structured datasets. They could automate the categorization of celestial objects such as galaxies, supernovae, and predict weather-related visibility constraints based on historical patterns (Solorio-Ramírez et al., 2023). Reinforcement learning is particularly valuable for dynamic environments where adaptability is key. These algorithms could dynamically adjust the telescope schedule based on real-time data like weather updates, the availability of instruments, and the emergence of transient astronomical events. Successful observation of a high-priority target could serve as a reward signal, guiding the algorithm toward optimal scheduling choices (Terranova et al., 2023). Moreover, predictive analytics, which employs statistical techniques to forecast based on current and historical data, could be used to predict optimal windows for observing specific celestial phenomena. In the quest to optimize the Rubin Observatory’s scheduling system, we identified the AI methodologies to address the unique challenges posed by its operational environment:

Supervised Machine Learning: Central to our consideration are neural networks and decision trees, renowned for their strong classification and predictive capabilities. With their deep learning power, Neural networks are adept at processing and making sense of the vast volumes of complex, high-dimensional data generated daily by the observatory (Caballero et al., 2020). On the other hand, decision trees offer a structured approach to decision-making, simplifying complex problems into more manageable parts by splitting data into branches based on decision points. These models are valuable for tasks like classifying different types of celestial bodies and predicting atmospheric visibility, leveraging past observation data to train highly accurate predictive models (Vy, Sen, & Santosh, 2021).

Reinforcement Learning: This approach is suited to environments that require continuous learning and adaptation. Reinforcement learning algorithms operate on the principle of action and feedback, adjusting real-time scheduling decisions based on immediate outcomes and long-term goals. This method’s adaptability makes it ideal for responding to unexpected changes in weather or discovering new transient events, where the system must quickly recalibrate its priorities and schedules to capture fleeting opportunities (Herrmann & Schaub, 2023).

Predictive Analytics: Using statistical techniques to interpret current and historical data, predictive analytics can accurately forecast future conditions. In the Rubin Observatory, this method anticipates optimal observing conditions, minimizing downtime and maximizing the telescope’s efficiency. Predictive analytics can integrate various data sources, including historical weather patterns and celestial event logs, to create a comprehensive predictive model that guides scheduling decisions (Ni et al., 2022).

Selection Rationale

Our ultimate choice of AI methodologies for the Rubin Observatory will hinge on a balance between benefits, the importance of explainability, and computational constraints. This decision process demonstrates our ability to judge AI’s rigor and select the most suitable technologies for this complex systems engineering problem. While machine learning algorithms like neural networks offer accuracy and efficiency, they can be difficult to interpret (Hložek et al., 2023). In contrast, while slightly less accurate, a decision tree model allows astronomers to visualize the logic behind the scheduling recommendations. The observatory’s need for dynamic adaptation to transient events and trust in the AI system will be crucial factors in the final selection (Johnson et al., 2021). Therefore, these are our reasons:

Balancing Complexity and Capability: While neural networks offer unparalleled accuracy and learning capabilities, their computational intensity and “black box” nature pose significant challenges, especially regarding interpretability and operational demands. On the other hand, decision trees provide a more straightforward, rule-based approach that is easier to understand and implement. This makes them particularly appealing for settings where decisions need to be transparent and justifiable to a broad scientific community (Li et al., 2022).

Ensuring Realtime Responsiveness: The dynamic nature of the observatory’s environment demands a system that can make decisions quickly and adapt its strategies based on real-time data. Reinforcement learning emerges as a powerful candidate here, given its ability to optimize decision-making continuously through trial and error, learning from each action’s consequences to refine future choices (Andersson, Heintz, & Doherty, 2015).

Integration and Trust: Integrating these AI methodologies into a cohesive system that astronomers trust and rely on requires a blend of these technologies. The system must perform well under varied conditions and align with the observatory’s operational protocols and data privacy standards. Therefore, AI solutions must demonstrate technical proficiency, alignment with ethical standards, and practical operability within the observatory’s infrastructure (Vilardell et al., 2016).

System Design and Development

The design and development of the AI-augmented telescope scheduling system for the Rubin Observatory emphasize adaptability, scalability, and seamless integration with the observatory’s complex data environment. The primary goal is to support real-time decision-making, enhance automation, and secure and strong operations (Thomas et al., 2020). Throughout this development phase, a focus on establishing the system’s core functionality will be balanced with the flexibility to accommodate future expansions as scientific objectives and technological capabilities evolve (Bianco et al., 2021).

System Architecture

The system’s modular architecture has clearly defined components and interfaces that facilitate flexibility and maintainability. Below are the primary components of the system:

Data Ingestion Module: This component collects and preprocesses data from diverse sources such as weather forecasts, astronomical databases, and direct inputs from telescope instrumentation. It protects the real-time or historical data formatted and ready for analysis (Baron, 2019).

AI Analysis Engine: At the heart of the system, this engine employs advanced machine learning algorithms to process the ingested data. It predicts optimal observing conditions and generates scheduling recommendations that are both timely and scientifically valuable (Voetberg & Nord, 2023).

Decision Support Interface: Designed with the end-user in mind, this interface presents the schedules proposed by the AI system in a user-friendly format. It allows astronomers to make informed decisions, manually overriding AI decisions if necessary, thereby maintaining essential human oversight (Frysak, 2017).

Integration Layer: This layer is where the AI system interfaces flawlessly with existing observatory infrastructure, such as telescope control systems, enhancing data exchange and operational coordination (Buur et al., 2016).

Monitoring and Feedback System: A continuous evaluation mechanism monitors the system’s performance, gathers user feedback, and provides actionable insights that drive system improvements. This feedback loop is crucial for iterative development and refinement (Stetzler et al., 2022).

Development Approach

The software development process will follow an agile methodology emphasizing rapid prototyping and continuous feedback from astronomers and stakeholders. This iterative approach helps the system align closely with the observatory’s real-world needs and remains adaptable to evolving scientific priorities. Initial development will center on creating a Minimum Viable Product (MVP), demonstrating the core AI-powered scheduling functionalities (A. Bulgarelli, 2019). For example, the MVP might prioritize observing supernovae based on weather forecasts and current telescope availability. This MVP will undergo rigorous testing in simulated environments before full deployment (J. Bento et al., 2022). Close collaboration between AI experts, software engineers, and domain experts in astronomy will be vital for success. This interdisciplinary approach will ensure that the AI system produces technically sound schedules, maximizes scientific opportunities, and integrates smoothly with the practical workflow of the observatory (Hanne Buur et al., 2016).

Objectives

This project exemplifies the practical application of our focus on AI and systems engineering. It provides an opportunity to move beyond theoretical concepts and address complex, real-world challenges within the demanding scientific environment of the Rubin Observatory. The project will also engage with current debates within AI-augmented systems. Securing the explainability of AI’s scheduling decisions is the most important goal. While a neural network model might initially deliver greater accuracy, a decision tree model will be used in the MVP. This allows for a clear visualization of the factors driving scheduling choices, which is key to building trust. Furthermore, the project will explore the optimal balance between algorithmic recommendations and astronomers’ irreplaceable expertise (Petrotta & Peterson, 2019). Designing, developing, and reflecting on this AI system will demonstrate a deep understanding of how AI can be successfully integrated into complex systems engineering projects. The analysis will highlight AI augmentation’s unique benefits and limitations in this context, providing valuable insights for future endeavors (Patil, 2023).

Explainability and Trust: The MVP’s use of a decision tree model underscores the project’s commitment to explainability and user trust. The system fosters greater acceptance and confidence in AI-driven decisions by allowing astronomers to visualize and understand the scheduling logic (GhoshRoy et al., 2022).

Balancing Expertise and AI: The project explores the delicate balance between leveraging AI’s capabilities and valuing human expertise. This balance is critical in ensuring the AI system enhances rather than replaces human decision-making processes (Ehsan et al., 2021).

Reflecting on AI Integration: The development and subsequent analysis of this AI system will provide valuable insights into the benefits and limitations of AI in systems engineering, contributing to ongoing discussions and future developments in the field (Pynadath et al., 2022).

Prototype/Simulation Development and Testing

Developing and testing a prototype is crucial for the Rubin Observatory’s AI-enhanced telescope scheduling system. This stage bridges the gap between theoretical concepts and practical implementation, allowing for a comprehensive assessment of the system’s performance and integration with existing infrastructure within a controlled environment. A solid simulation environment will be fundamental for testing the prototype (Xu et al., 2020). This simulation will model realistic observing conditions, including weather patterns, unexpected celestial events, and fluctuations in telescope availability. By confronting the AI system with these dynamic scenarios, developers can refine the algorithms and settings for optimal real-world performance (González et al., 2018). Emphasis will be placed on ensuring the reliability and efficiency of the system before its deployment in the live observatory environment.

The prototype development will follow an iterative approach. Feedback from each testing cycle will drive improvements in AI functionality and user interface design. Addressing issues early in the process will minimize the risk of costly changes later in the project (Peterson et al., 2015).

Implementation Details

The prototype implements core components designed during the system architecture phase. The AI Analysis Engine, which processes diverse data streams to generate optimized scheduling suggestions, is central to this. This engine works in tandem with the Data Ingestion Module, configured to seamlessly assimilate real-time and historical data, thus ensuring that the AI can access comprehensive inputs needed for accurate decision-making (Patil, 2023). A user-friendly Decision Support Interface is developed to allow astronomers to interact with the AI’s recommendations easily. This interface supports manual overrides, ensuring that human expertise continues to play a critical role in the decision-making process. Integration testing is rigorously conducted to facilitate effective communication among these components and the existing IT infrastructure at the observatory (Stetzler et al., 2022). An initial version of the Monitoring and Feedback System is also implemented to track the AI system’s performance and gather user feedback. This feedback is invaluable, providing insights crucial for refining the AI functionalities and aligning them more closely with user needs and system requirements. Python, known for its strong libraries and frameworks suited to AI and data processing, is selected as the primary programming language, supporting the development of a flexible and powerful system (Robitaille et al., 2013).

Testing Strategy

The testing strategy is crafted to evaluate the AI system against stringent performance metrics defined by the observatory’s operational requirements. These metrics include the increased detection rate of high-priority transient events, the reduction in the time astronomers spend scheduling observations, and the overall enhancement of the scientific output from telescope time (Colomé et al., 2012). The effectiveness of the AI system is compared with traditional scheduling methods, providing a clear measure of the added value brought by AI integration. While simulation forms a core part of the testing phase, acknowledging the limitations of a purely virtual environment is crucial (Schussler et al., 2023). Therefore, selective live testing sessions are planned, during which the AI system will interact with actual data flows and operational conditions at the observatory. This step is important for testing the system’s functionality in a real-world setting and identifying and mitigating possible security vulnerabilities (Lsst Science Collaborations et al., 2017).

Results

Prototype testing is expected to reveal the challenges of integrating AI into complex existing systems, including data interoperability issues, trade-offs between AI techniques such as accuracy vs. explainability, and ongoing user training. Addressing these challenges is vital for successfully adopting AI technologies (Cotton, 2008). The analysis of the AI solutions must also scrutinize the system to introduce biases into scientific research. Training AI primarily on historical data risks unintentionally perpetuating existing biases. Understanding these risks and actively developing strategies to mitigate biases will ensure that the AI system equitably enhances the observatory’s mission and facilitates new rather than reinforced scientific pathways (Kremer et al., 2017). Ultimately, the project’s success extends beyond technical achievements into human-AI collaboration. Lessons learned throughout development and testing regarding astronomer preferences, trust in the AI’s recommendations, and the evolving role of humans within an AI-augmented system will be invaluable. These findings can pave the way for the smooth integration of AI into established scientific settings and inform future project endeavors (Dee, 2005).

Discussion

As we look to the future, this project makes us think about how AI technologies could have even more transformative power as they improve. Creating AI systems that find strange or unexpected events could speed up chance findings and expand our understanding of the universe (Kremer et al., 2017). In addition, AI could free up scientists to spend more time on creative scientific work and new research by eliminating boring tasks and simplifying data analysis processes. A new generation of scientific discovery made possible by AI-powered tools that anyone with internet access and curiosity can use could be fostered by bringing the lessons learned from this project to any science efforts, which could vastly increase public involvement in astronomy (Cotton, 2008).

Key Findings

The AI system made scheduling much more efficient. It can dynamically adapt to changing conditions and set priorities for observations based on real-time data from a camera that has been measured and adjusted. This is shown by the fact that the AI could find a short weather window that lets people see an astronomical phenomenon by going out of view. Using the old ways of scheduling would have missed these kinds of findings. The AI has directly affected the observatory’s main goal, as shown by the higher scientific findings and better data collection (Patil, 2023). The AI system also did a great job of handling the observatory’s huge and complicated information. Its accurate processing and analysis of astronomical data sped up the discovery of new scientific ideas and made analysis easier. During the testing and development stages, the project taught us a lot about how hard it is to add AI to a current system (Solar & Atkinson, 2020). For future AI deployments in large-scale scientific systems to work, problems like data compatibility and the need for ongoing user training will have to be solved (Johnston & Miller, 1989).

Importantly, the project emphasized balancing automation and human control within AI-augmented systems. Even though AI was better at some scheduling tasks, scientists’ deep knowledge of the subject was still needed to understand complicated scientific goals and spot oddities that could mean things that haven’t been seen before. This shows how important it is for humans and AI to work together and offers a model in which AI is a powerful decision-support tool and scientists are in charge (Frank & Kürklü, 2005).

Impact and Implications

The Rubin Observatory’s introduction of AI has many scientific and operational implications. The observatory has been improved to properly use larger data sets and quickly respond to changing worldwide events, increasing its input to astronomical research projects. AI-driven scheduling improves resource management, which could lower costs and increase the observatory’s long-term viability. The AI’s ability to see into the future also lets proactive telescope repair happen, protecting valuable viewing time (Johnston & Miller, 1989). The project also demonstrates the prospects and difficulties of AI-enabled science. This statement draws attention to the current talks about the possibility of bias in AI decision-making, the need for open and understandable algorithms, and the importance of protecting data privacy in astronomical research. It is key to effectively address these ethics issues linked to AI systems to support fair and reasonable improvement of scientific progress. The information gathered from this project will shape future discussions and direct the development of the best possible methods as AI spreads throughout the scientific community (Solar & Atkinson, 2020). AI allows astronomical findings to be available to a bigger audience, which is also suggested by this project, in addition to its effect on the Rubin Observatory. Tools that AI drives may be available to smaller observatories, educational institutions, and even regular people worldwide. This can make data analysis more accessible to a bigger audience, which could lead to new findings and increased interest in science among the general public. A new era of finding driven by citizen science is now possible thanks to AI’s capacity to find patterns in large data sets, despite astronomers’ incomplete understanding of such patterns. These days, expert and hobbyist astronomers use AI systems to learn more and more (Patil, 2023).

Applying Knowledge

This project shows how systems engineering ideas can be used in the real world in an AI-driven setting. MBSE methods were very helpful for dealing with complexity and making it easier for astronomers and technical experts to talk to each other. Working together was crucial for ensuring the AI system fit the observatory’s specific scientific and practical needs. The project shows that systems engineering methods can be successfully changed to work with quickly evolving technologies like AI. This gives us a good starting point for future work (Karban et al., 2014). The project also reinforces the importance of responsible AI development. It was necessary to prioritize explainability to get people to believe the AI system, especially when choosing between a more complicated “black box” model and a simpler but explainable one. Regular performance tracking and data security measures were built into the design process to keep things honest and lower risks. A human-centered design method was stressed throughout the project. Astronomers were involved in every step of the development process, from figuring out what was needed to check the prototype. Their feedback allowed the AI system’s features and interface design to align with how astronomers work and with scientific goals. This focus on the user experience will still be important once the system is fully integrated into the observatory’s work (Riva et al., 2022). The project also knows that adding AI to a complicated system is a process that happens over time, only some at a time. A flexible framework and a Monitoring and Feedback System were important parts of the design and development. This will make it possible to keep improving AI programs based on data from the real world and changing scientific goals. It emphasizes the importance of flexibility and that successful AI integration necessitates dedication to ongoing evaluation, improvement, and user training (Boy, 2021).

Mastery of Objective

This project makes it clear that methods for systems engineering need to change along with technologies like AI that are improving quickly. To secure ethical and scientifically sound systems, it emphasizes the importance of looking into the dependability, openness, and flaws of AI-driven decision-making. Dealing with problems like combining data, keeping an eye on the system, and the need to train users all the time shows how hard it is to use AI in real life. The project makes a strong case for a systems engineering approach that values technological innovation and adapting human skills and organizational structures to include AI’s smooth and reliable integration into scientific processes (Kasabov, 2005). As AI technologies advance, this project makes one consider the ability for even more transformative power. The development of AI systems, which are intended especially to find unexpected or strange events, can speed up random discoveries and push the limits of astronomical knowledge. In addition, AI helps astronomers focus on creative scientific work and new research by eliminating repetitive tasks and simplifying data analysis processes. A new scientific discovery powered by AI-powered tools available to anyone interested in and with internet access could be fostered by bringing the lessons learned from this project to citizen science efforts, which could vastly increase public involvement in astronomy (Xu et al., 2023). This project also highlights the importance of ongoing learning and adaptation for AI systems and the human experts who work with them. Astronomers’ comments and thoughts will be crucial in improving the system to boost scientific output further as they gain experience using AI tools. At the same time, the AI system needs to be improved and updated as new data trends appear and scientific knowledge grows. As this dynamic interaction between human knowledge and AI-powered research shows, we need systems engineering methods to make these ongoing co-developments possible (Lase & Nkosi, 2023).

Conclusion

AI can catalyze new forms of scientific inquiry and public engagement in astronomy. By extending the reach of AI tools to citizen science platforms, we can democratize access to astronomical research, fostering a new era of community-driven scientific exploration. This project not only advances the operational capabilities of the Rubin Observatory but also contributes to a broader dialogue about the role of AI in shaping the future of scientific discovery and public participation in science.

References

11.0 Ground Data Systems and Mission Operations — NASA. (n.d.). Retrieved May 7, 2024, from https://www.nasa.gov/smallsat-institute/sst-soa/ground-data-systems-and-mission-operations/

Adaptive optics control using model-based reinforcement learning. (n.d.). Retrieved May 6, 2024, from https://opg.optica.org/oe/fulltext.cfm?uri=oe-29-10-15327&id=450708

Albrecht, R. (1989). Artificial Intelligence: What can it do for Astronomy? In V. Di Gesù, L. Scarsi, P. Crane, J. H. Friedman, S. Levialdi, & M. C. Maccarone (Eds.), Data Analysis in Astronomy III (pp. 191–204). Springer US. https://doi.org/10.1007/978-1-4684-5646-2_22

Alves, C. S., Peiris, H. V., Lochner, M., McEwen, J. D., Allam, T., Biswas, R., & Collaboration, T. L. D. E. S. (2022). Considerations for Optimizing the Photometric Classification of Supernovae from the Rubin Observatory. The Astrophysical Journal Supplement Series, 258(2), 23. https://doi.org/10.3847/1538-4365/ac3479

An integrated system for astronomical telescope based on stellarium | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/6016447

Analysis: How AI is helping astronomers study the universe | PBS NewsHour. (n.d.). Retrieved May 7, 2024, from https://www.pbs.org/newshour/science/analysis-how-ai-is-helping-astronomers-study-the-universe

Analyzing and Processing of Astronomical Images using Deep Learning Techniques | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/9622583

Andersson, O., Heintz, F., & Doherty, P. (2015). Model-Based Reinforcement Learning in Continuous Environments Using Real-Time Constrained Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), Article 1. https://doi.org/10.1609/aaai.v29i1.9623

Andolfato, L., Karban, R., Schilling, M., Sommer, H., Zamparelli, M., & Chiozzi, G. (2014). Experiences in Applying Model Driven Engineering to the Telescope and Instrument Control System Domain. In J. Dingel, W. Schulte, I. Ramos, S. Abrahão, & E. Insfran (Eds.), Model-Driven Engineering Languages and Systems (pp. 403–419). Springer International Publishing. https://doi.org/10.1007/978-3-319-11653-2_25

Artificial Intelligence Approaches to Astronomical Observation Scheduling | Semantic Scholar. (n.d.). Retrieved May 6, 2024, from https://www.semanticscholar.org/paper/Artificial-Intelligence-Approaches-to-Astronomical-Johnston-Miller/55bac94d24a676f0aa6f20144a9210a2e82eb96f

Artificial intelligence for improved fitting of trajectories of elementary particles in dense materials immersed in a magnetic field | Communications Physics. (n.d.). Retrieved May 7, 2024, from https://www.nature.com/articles/s42005-023-01239-4

Bai, Y., Liu, J., Wang, S., & Yang, F. (2019). Machine learning Applied to Star-Galaxy-QSO Classification and Stellar Effective Temperature Regression. The Astronomical Journal, 157(1), 9. https://doi.org/10.3847/1538-3881/aaf009

Baron, D. (2019). Machine Learning in Astronomy: A practical overview. arXiv: Instrumentation and Methods for Astrophysics. https://www.semanticscholar.org/paper/Machine-Learning-in-Astronomy%3A-a-practical-overview-Baron/75c00a85f820c9d012bdb1a54a210c265d7c20bc

Bellm, E. C., Ford, E. B., Tohuvavohu, A., Coughlin, M. W., Morris, B., Miller, B., Sobeck, J., Riddle, R., Dong, C., & Yoachim, P. (2019). Scheduling Discovery in the 2020s (arXiv:1907.07817). arXiv. http://arxiv.org/abs/1907.07817

Bento, J., Arnold, D. M., Smith, R. J., Fernández-Valdivia, J. J., Gil, J. L., Martin, J. B., & Gill, M. A. T. (2022). Experience of utilising CI/CD practices in the development of software for a modern astronomical observatory. Software and Cyberinfrastructure for Astronomy VII, 12189, 48–53. https://doi.org/10.1117/12.2629975

Bhagyashree Patil. (2023). AI in Astronomy. International Journal of Advanced Research in Science, Communication and Technology, 476–485. https://doi.org/10.48175/IJARSCT-12167

Bianco, F., Ivezić, Ž., Jones, R. L., Graham, M., Marshall, P., Saha, A., Strauss, M., Yoachim, P., Ribeiro, T., Anguita, T., Bauer, A., Bauer, F., Bellm, E., Blum, R., Brandt, W., Brough, S., Catelán, M., Clarkson, W., Connolly, A., … Willman, B. (2021). Optimization of the Observing Cadence for the Rubin Observatory Legacy Survey of Space and Time: A Pioneering Process of Community-focused Experimental Design. The Astrophysical Journal Supplement Series, 258. https://doi.org/10.3847/1538-4365/ac3e72

Bias and data assimilation — Dee — 2005 — Quarterly Journal of the Royal Meteorological Society — Wiley Online Library. (n.d.). Retrieved May 6, 2024, from https://rmets.onlinelibrary.wiley.com/doi/10.1256/qj.05.137

Big Universe, Big Data: Machine Learning and Image Analysis for Astronomy | IEEE Journals & Magazine | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/7887648

Blanco-Justicia, A., Domingo-Ferrer, J., Martínez, S., & Sánchez, D. (2020). Machine learning explainability via microaggregation and shallow decision trees. Knowledge-Based Systems, 194, 105532. https://doi.org/10.1016/j.knosys.2020.105532

Boy, G. A., & Boy, G. A. (2021). Model-Based Human Systems Integration Flexibility. 59–72. https://doi.org/10.1007/978-3-030-76391-6_6

Bulgarelli, A. (2019). The AGILE Gamma-Ray observatory: Software and pipelines. Experimental Astronomy, 48(2), 199–231. https://doi.org/10.1007/s10686-019-09644-w

Buur, H., Subramaniam, A., Gillies, K., Dumas, C., & Bhatia, R. (2016). TMT approach to observatory software development process. Software and Cyberinfrastructure for Astronomy IV, 9913, 491–500. https://doi.org/10.1117/12.2234102

ChatGPT. (n.d.). Retrieved May 5, 2024, from https://chatgpt.com/?oai-dm=1

Collaboration, L. D. E. S. (n.d.). Home. LSST Dark Energy Science Collaboration. Retrieved May 7, 2024, from https://lsstdesc.org/

Connor, L., & Leeuwen, J. van. (2018). Applying Deep Learning to Fast Radio Burst Classification. The Astronomical Journal, 156(6), 256. https://doi.org/10.3847/1538-3881/aae649

Cowing, K. (2023, February 24). Analyzing Astronomical Data with Machine Learning Techniques. SpaceRef. https://spaceref.com/newspace-and-tech/analyzing-astronomical-data-with-machine-learning-techniques/

Data Mining and Machine Learning in Astronomy — Nicholas M. Ball & Robert J. Brunner. (n.d.). Retrieved May 7, 2024, from https://ned.ipac.caltech.edu/level5/March11/Ball/Ball3.html

de Dios Rojas Olvera, J., Gómez-Vargas, I., & Vázquez, J. A. (2022). Observational Cosmology with Artificial Neural Networks. Universe, 8(2), Article 2. https://doi.org/10.3390/universe8020120

Deep Neural Network Initialization With Decision Trees | IEEE Journals & Magazine | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/8478232

Dobrzycki, A., Delmotte, N., Rossat, N., Pirenne, B., Avelans, C., & Rainer, N. (2004). Data interface control at the European Southern Observatory. Optimizing Scientific Return for Astronomy through Information Technologies, 5493, 117–125. https://doi.org/10.1117/12.551423

Electronics | Free Full-Text | Explainable AI to Predict Male Fertility Using Extreme Gradient Boosting Algorithm with SMOTE. (n.d.). Retrieved May 6, 2024, from https://www.mdpi.com/2079-9292/12/1/15

Erasmus, D. A., & Sarazin, M. S. (2001). Forecasting precipitable water vapor and cirrus cloud cover for astronomical observatories: Satellite image processing guided by synoptic model dissemination data. Remote Sensing of Clouds and the Atmosphere V, 4168, 317–328. https://doi.org/10.1117/12.413848

ESO — AIA2019. (n.d.). Retrieved May 7, 2024, from https://www.eso.org/sci/meetings/2019/AIA2019.html

Evolving systems. (2001). In Special Matrices of Mathematical Physics (pp. 21–23). WORLD SCIENTIFIC. https://doi.org/10.1142/9789812799838_0002

Expanding Explainability: Towards Social Transparency in AI systems | Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. (n.d.). Retrieved May 6, 2024, from https://dl.acm.org/doi/10.1145/3411764.3445188

Explainable Reinforcement Learning in Human-Robot Teams: The Impact of Decision-Tree Explanations on Transparency | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/9900608

Frank, J., & Kürklü, E. (2005). Mixed Discrete and Continuous Algorithms for Scheduling Airborne Astronomy Observations. In R. Barták & M. Milano (Eds.), Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems (pp. 183–200). Springer. https://doi.org/10.1007/11493853_15

Frysak, J. (2017). Feedback Mechanisms for Decision Support Systems: A Literature Review. In Á. Rocha, A. M. Correia, H. Adeli, L. P. Reis, & S. Costanzo (Eds.), Recent Advances in Information Systems and Technologies (pp. 481–490). Springer International Publishing. https://doi.org/10.1007/978-3-319-56538-5_49

‎Gemini — Chat to supercharge your ideas. (n.d.). Gemini. Retrieved May 5, 2024, from https://gemini.google.com

Giordano, C., Rafalimanana, A., Ziad, A., Aristidi, E., Chabé, J., Fanteï-Caujole, Y., & Renaud, C. (2021). Contribution of statistical site learning to improve optical turbulence forecasting. Monthly Notices of the Royal Astronomical Society, 504(2), 1927–1938. https://doi.org/10.1093/mnras/staa3709

Gonzalez, R., Alvarez, O., & Cristancho, D. (2018). Developing a Prototype for Remote Viewing of Telescope Images Using Mobile Devices (p. 5). https://doi.org/10.1109/ANDESCON.2018.8564609

Guy, L. P., Bellm, E., Blum, B., Graham, M. L., Ivezić, Ž., & Strauss, M. (2021). Rubin Observatory Plans for an Early Science Program. https://doi.org/10.5281/zenodo.5683849

Hložek, R., Malz, A. I., Ponder, K. A., Dai, M., Narayan, G., Ishida, E. E. O., Jr, T. A., Bahmanyar, A., Bi, X., Biswas, R., Boone, K., Chen, S., Du, N., Erdem, A., Galbany, L., Garreta, A., Jha, S. W., Jones, D. O., Kessler, R., … Zuo, W. (2023). Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC). The Astrophysical Journal Supplement Series, 267(2), 25. https://doi.org/10.3847/1538-4365/accd6a

Human-Centric AI: Understanding and Enhancing Collaboration between Humans and Intelligent Systems | Algorithm Asynchronous. (n.d.). Retrieved May 6, 2024, from https://hasmed.org/index.php/jourasy/article/view/49

Information | Free Full-Text | Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence. (n.d.). Retrieved May 7, 2024, from https://www.mdpi.com/2078-2489/11/4/193

Integrated photonic building blocks for next-generation astronomical instrumentation II: the multimode to single mode transition. (n.d.). Retrieved May 6, 2024, from https://opg.optica.org/oe/fulltext.cfm?uri=oe-21-22-27197&id=274057

Jeddi, A. B., Dehghani, N. L., & Shafieezadeh, A. (2021). Lyapunov-based uncertainty-aware safe reinforcement learning. ArXiv. https://www.semanticscholar.org/paper/Lyapunov-based-uncertainty-aware-safe-reinforcement-Jeddi-Dehghani/ecc368ca0bd209466a23b86af17fbc187f3a0d29

Johnson, M. A. C., Paradies, M., Dembska, M., Lackeos, K., Klöckner, H.-R., Champion, D., & Schindler, S. (2021). Astronomical Pipeline Provenance: A Use Case Evaluation. ArXiv. https://www.semanticscholar.org/paper/Astronomical-Pipeline-Provenance%3A-A-Use-Case-Johnson-Paradies/683cdb46b32d5fed8f39438a2a3b65f3e83311fe

Karban, R., Andolfato, L., Bristow, P., Chiozzi, G., Esselborn, M., Schilling, M., Schmid, C., Sommer, H., & Zamparelli, M. (2014). Model based systems engineering for astronomical projects. Modeling, Systems Engineering, and Project Management for Astronomy VI, 9150, 208–222. https://doi.org/10.1117/12.2055540

Kembhavi, A., & Pattnaik, R. (2022). Machine learning in astronomy. Journal of Astrophysics and Astronomy, 43(2), 76. https://doi.org/10.1007/s12036-022-09871-2

Kennamer, N., Ishida, E. E. O., González-Gaitán, S., Souza, R. S. de, Ihler, A., Ponder, K., Vilalta, R., Möller, A., Jones, D. O., Dai, M., Krone-Martins, A., Quint, B., Sreejith, S., Malz, A. I., & Galbany, L. (2020). Active learning with RESSPECT: Resource allocation for extragalactic astronomical transients. 2020 IEEE Symposium Series on Computational Intelligence (SSCI), 3115–3124. https://doi.org/10.1109/SSCI47803.2020.9308300

King, O., Blinov, D., Ramaprakash, A., Myserlis, I., Angelakis, E., Baloković, M., Feiler, R., Fuhrmann, L., Hovatta, T., Khodade, P., Kougentakis, A., Kylafis, N., Kus, A., Paleologou, E., Panopoulou, G., Papadakis, I., Papamastorakis, I., Paterakis, G., Pavlidou, V., & Zensus, J. A. (2013). The RoboPol Pipeline and Control System. Monthly Notices of the Royal Astronomical Society, 442. https://doi.org/10.1093/mnras/stu176

Krenn, M., Pollice, R., Guo, S. Y., Aldeghi, M., Cervera-Lierta, A., Friederich, P., dos Passos Gomes, G., Häse, F., Jinich, A., Nigam, A., Yao, Z., & Aspuru-Guzik, A. (2022). On scientific understanding with artificial intelligence. Nature Reviews Physics, 4(12), 761–769. https://doi.org/10.1038/s42254-022-00518-3

Learning agents for uncertain environments (extended abstract) | Proceedings of the eleventh annual conference on Computational learning theory. (n.d.). Retrieved May 6, 2024, from https://dl.acm.org/doi/10.1145/279943.279964

Leikanger, P. R. (2019). Modular RL for Real-Time Learning in Physical Environments. 2019 Conference on Cognitive Computational Neuroscience. 2019 Conference on Cognitive Computational Neuroscience. https://doi.org/10.32470/CCN.2019.1270-0

Liu, S., Duffy, A. H. B., Whitfield, R. I., & Boyle, I. M. (2010). Integration of decision support systems to improve decision support performance. Knowledge and Information Systems, 22(3), 261–286. https://doi.org/10.1007/s10115-009-0192-4

Liu, X., Wang, X., & Matwin, S. (2018). Improving the Interpretability of Deep Neural Networks with Knowledge Distillation. 2018 IEEE International Conference on Data Mining Workshops (ICDMW), 905–912. https://doi.org/10.1109/ICDMW.2018.00132

LSST Camera. (n.d.). SLAC National Accelerator Laboratory. Retrieved May 7, 2024, from https://www6.slac.stanford.edu/lsst

Machine Learning in Astronomy:. A Case Study in Quasar-Star… | by Rishabh | Medium. (n.d.). Retrieved May 7, 2024, from https://rbrishabh76.medium.com/machine-learning-in-astronomy-724b1da1619f

Machine Learning Tools Automatically Classify 1,000 Supernovae. (2022, November 22). California Institute of Technology. https://www.caltech.edu/about/news/machine-learning-tools-automatically-classify-1000-supernovae

Marshall, P., Clarkson, W., Shemmer, O., Biswas, R., Val-Borro, M. de, Rho, J., Jones, L., Anguita, T., Ridgway, S., Bianco, F., Ivezic, Z., Lochner, M., Meyers, J., Vivas, K., Graham, M., Claver, C., Digel, S., Kasliwal, V., McGehee, P. M., … Awan, H. (2017). LSST Science Collaborations Observing Strategy White Paper: “Science-driven Optimization of the LSST Observing Strategy” (v1.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.842713

Marshall, P. J., Lintott, C. J., & Fletcher, L. N. (2015). Ideas for Citizen Science in Astronomy. Annual Review of Astronomy and Astrophysics, 53(Volume 53, 2015), 247–278. https://doi.org/10.1146/annurev-astro-081913-035959

Martinelli, A. (2012). Vision and IMU Data Fusion: Closed-Form Solutions for Attitude, Speed, Absolute Scale, and Bias Determination. IEEE Transactions on Robotics, 28(1), 44–60. https://doi.org/10.1109/TRO.2011.2160468

Martínez, L. A., Villarreal, J. L., Ángeles, F., & Bernal, A. (2010). A virtual reality environment for telescope operation. 7740, 77402B. https://doi.org/10.1117/12.856925

Masci, F. J., Laher, R. R., Rusholme, B., Shupe, D. L., Groom, S., Surace, J., Jackson, E., Monkewitz, S., Beck, R., Flynn, D., Terek, S., Landry, W., Hacopians, E., Desai, V., Howell, J., Brooke, T., Imel, D., Wachter, S., Ye, Q.-Z., … Kulkarni, S. R. (2018). The Zwicky Transient Facility: Data Processing, Products, and Archive. Publications of the Astronomical Society of the Pacific, 131(995), 018003. https://doi.org/10.1088/1538-3873/aae8ac

Meher, S. K., & Panda, G. (2021). Deep learning in astronomy: A tutorial perspective. The European Physical Journal Special Topics, 230(10), 2285–2317. https://doi.org/10.1140/epjs/s11734-021-00207-9

Miller, G. (1989). Artificial intelligence applications for Hubble Space Telescope operations. In A. Heck & F. Murtagh (Eds.), Knowledge-Based Systems in Astronomy (pp. 3–31). Springer. https://doi.org/10.1007/3-540-51044-3_14

Naul, B., Bloom, J. S., Pérez, F., & van der Walt, S. (2018). A recurrent neural network for classification of unevenly sampled variable stars. Nature Astronomy, 2(2), 151–155. https://doi.org/10.1038/s41550-017-0321-z

Neira, M., Gómez, C., Suárez-Pérez, J. F., Gómez, D. A., Reyes, J. P., Hoyos, M. H., Arbeláez, P., & Forero-Romero, J. E. (2020). MANTRA: A Machine-learning Reference Light-curve Data Set for Astronomical Transient Event Recognition. The Astrophysical Journal Supplement Series, 250(1), 11. https://doi.org/10.3847/1538-4365/aba267

Neural Decision Tree: A New Tool for Building Forecast Models for Plasmasphere Dynamics — Lu — 2022 — Earth and Space Science — Wiley Online Library. (n.d.). Retrieved May 6, 2024, from https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2021EA002175

Ni, W.-J., Shen, Q.-L., Zeng, Q.-T., Wang, H.-Q., Cui, X.-Q., & Liu, T. (2022). Data-driven Seeing Prediction for Optics Telescope: From Statistical Modeling, Machine Learning to Deep Learning Techniques. Research in Astronomy and Astrophysics, 22(12), 125003. https://doi.org/10.1088/1674-4527/ac977b

Norris, R. P. (2017). Discovering the Unexpected in Astronomical Survey Data. Publications of the Astronomical Society of Australia, 34, e007. https://doi.org/10.1017/pasa.2016.63

Obit: A Development Environment for Astronomical Algorithms | Semantic Scholar. (n.d.). Retrieved May 6, 2024, from https://www.semanticscholar.org/paper/Obit%3A-A-Development-Environment-for-Astronomical-Cotton/d61db11a336e236178f7e56089f9265e644cffb0

Patel, M. R. (2022). HOW DOES ARTIFICIAL INTELLIGENCE HELP ASTRONOMY? A REVIEW. IJRDO -Journal of Computer Science Engineering, 8(11), Article 11. https://doi.org/10.53555/cse.v8i11.5427

Peterson, J. R., Jernigan, J. G., Kahn, S. M., Rasmussen, A. P., Peng, E., Ahmad, Z., Bankert, J., Chang, C., Claver, C., Gilmore, D. K., Grace, E., Hannel, M., Hodge, M., Lorenz, S., Lupu, A., Meert, A., Nagarajan, S., Todd, N., Winans, A., & Young, M. (2015). SIMULATION OF ASTRONOMICAL IMAGES FROM OPTICAL SURVEY TELESCOPES USING A COMPREHENSIVE PHOTON MONTE CARLO APPROACH. The Astrophysical Journal Supplement Series, 218(1), 14. https://doi.org/10.1088/0067-0049/218/1/14

Petrotta, M., & Peterson, T. (2019). Implementing Augmented Intelligence In Systems Engineering. INCOSE International Symposium, 29(1), 543–543. https://doi.org/10.1002/j.2334-5837.2019.00619.x

QuillBot: Make Writing Painless. (n.d.). Retrieved May 5, 2024, from https://quillbot.com

Rafalimanana, A., Giordano, C., Ziad, A., & Aristidi, E. (2022). Optimal Prediction of Atmospheric Turbulence by Means of the Weather Research and Forecasting Model. Publications of the Astronomical Society of the Pacific, 134(1035), 055002. https://doi.org/10.1088/1538-3873/ac6536

Raschka, S., Patterson, J., & Nolet, C. (2020). Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence. Information, 11(4), Article 4. https://doi.org/10.3390/info11040193

Reinforcement Learning for the Agile Earth-Observing Satellite Scheduling Problem | IEEE Journals & Magazine | IEEE Xplore. (n.d.). Retrieved May 6, 2024, from https://ieeexplore.ieee.org/document/10058020

Reis, I., Rotman, M., Poznanski, D., Prochaska, J. X., & Wolf, L. (2021). Effectively using unsupervised machine learning in next generation astronomical surveys. Astronomy and Computing, 34, 100437. https://doi.org/10.1016/j.ascom.2020.100437

Research on schedulers for astronomical observatories — NASA/ADS. (n.d.). Retrieved May 6, 2024, from https://ui.adsabs.harvard.edu/abs/2012SPIE.8448E..1LC/abstract

Riva, M. M., Balestra, A., Zanutta, A., Xompero, M., Genoni, M., Briguglio, R., Scalera, M. A., Dinuzzi, G., & Fierro, D. (2022). Astro MBSE: Model based system engineering synthesized for the Italian astronomical community. In G. Z. Angeli & P. Dierickx (Eds.), Modeling, Systems Engineering, and Project Management for Astronomy X (p. 61). SPIE. https://doi.org/10.1117/12.2630392

Robitaille, T. P., Tollerud, E. J., Greenfield, P., Droettboom, M., Bray, E., Aldcroft, T., Davis, M., Ginsburg, A., Price-Whelan, A. M., Kerzendorf, W. E., Conley, A., Crighton, N., Barbary, K., Muna, D., Ferguson, H., Grollier, F., Parikh, M. M., Nair, P. H., Günther, H. M., … Streicher, O. (2013). Astropy: A community Python package for astronomy. Astronomy & Astrophysics, 558, A33. https://doi.org/10.1051/0004-6361/201322068

Rooyen, R. van, Maartens, D. S., & Martinez, P. (2018). Autonomous observation scheduling in astronomy. Observatory Operations: Strategies, Processes, and Systems VII, 10704, 393–408. https://doi.org/10.1117/12.2311839

Rubin Observatory Legacy Survey of Space and Time (LSST) | ESCAPE. (n.d.). Retrieved May 7, 2024, from https://projectescape.eu/science-projects/rubin-observatory-legacy-survey-space-and-time-lsst

Sample manuscript showing specifications and style. (n.d.). Retrieved May 7, 2024, from https://docushare.lsst.org/docushare/dsweb/GetRendition/Publication-118/html

Schüssler, F., Ashkar, H., Bernardini, E., Berti, A., Bradascio, F., Buson, S., Dorner, D., Jin, W., Kukec Mezek, G., Santander, M., Satalecka, K., Schleicher, B., Senniappan, M., Viale, I., IceCube, Abbasi, R., Ackermann, M., Adams, J., Agarwalla, S. K., … Yamamoto, T. (2023). Joint searches by FACT, H.E.S.S., MAGIC and VERITAS for VHE gamma-ray emission associated with neutrinos detected by IceCube. In Proceedings of 38th International Cosmic Ray Conference — PoS(ICRC2023) (Vol. 444, p. 1501). SISSA Medialab. https://doi.org/10.22323/1.444.1501

Sebag, J., Claver, C. F., Thomas, S. J., Reil, K., Barr, J., Bechtol, K., Daruich, F., Fabrega, J., Guy, L. P., Ingraham, P., Krabbendam, V., Lopez, M., Munoz, F., Neill, D., O’Mullane, W., Reuter, M., Roberts, A., Ribeiro, T., Serrano, E., … Xin, B. (2020). Vera C. Rubin Observatory system integration, test, and commissioning: Strategy and status. Ground-Based and Airborne Telescopes VIII, 11445, 396–415. https://doi.org/10.1117/12.2561675

Sheth, A., Gaur, M., Roy, K., & Faldu, K. (2021). Knowledge-Intensive Language Understanding for Explainable AI. IEEE Internet Computing, 25(5), 19–24. https://doi.org/10.1109/MIC.2021.3101919

Solorio-Ramírez, J.-L., Jiménez-Cruz, R., Villuendas-Rey, Y., & Yáñez-Márquez, C. (2023). Random forest Algorithm for the Classification of Spectral Data of Astronomical Objects. Algorithms, 16(6), Article 6. https://doi.org/10.3390/a16060293

Song, Y., Zhu, Y., Nan, T., Hou, J., Du, S., & Song, S. (2020). Accelerating Faceting Wide-Field Imaging Algorithm with FPGA for SKA Radio Telescope as a Vast Sensor Array. Sensors, 20(15), Article 15. https://doi.org/10.3390/s20154070

Spiral Model — Software Engineering. (2018, March 26). GeeksforGeeks. https://www.geeksforgeeks.org/software-engineering-spiral-model/

Sponsler, J., Johnston, M., Miller, G., Krueger, A., Lucks, M., & Giuliano, M. (1991). An AI scheduling environment for the Hubble Space Telescope. https://doi.org/10.2514/6.1991-3703

Sr, M. S., & Atkinson, J. (2020). Efficient astronomy scheduling using artificial immune systems. Observatory Operations: Strategies, Processes, and Systems VIII, 11449, 298–311. https://doi.org/10.1117/12.2563632

Stetzler, S., Jurić, M., Boone, K., Connolly, A., Slater, C. T., & Zečević, P. (2022). The Astronomy Commons Platform: A Deployable Cloud-based Analysis Platform for Astronomy. The Astronomical Journal, 164(2), 68. https://doi.org/10.3847/1538-3881/ac77fb

Telescope Control Tutorial. (n.d.). Retrieved May 7, 2024, from https://cdn.diffractionlimited.com/help/maximdl/Basic_Control.htm

Telescope, L. S. S. (n.d.). Rubin Observatory. Rubin Observatory. Retrieved May 7, 2024, from https://www.lsst.org/

Telescopius — Astronomy Planning Made Easy. (n.d.). Retrieved May 7, 2024, from https://www.insightobservatory.com/2020/04/telescopius-astronomy-planning-made-easy.html

Terranova, F., Voetberg, M., Nord, B., & Pagul, A. (2023). Self-Driving Telescopes: Autonomous Scheduling of Astronomical Observation Campaigns with Offline Reinforcement Learning (arXiv:2311.18094). arXiv. https://doi.org/10.48550/arXiv.2311.18094

The Basics for Astrophysics Machine Learning: A general overview | LinkedIn. (n.d.). Retrieved May 7, 2024, from https://www.linkedin.com/pulse/basics-astrophysics-machine-learning-general-overview-yan-barros-a7yff/

The research of intelligentization of control systems for large astronomical optical telescopes. (n.d.). Retrieved May 6, 2024, from https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11445/2576213/The-research-of-intelligentization-of-control-systems-for-large-astronomical/10.1117/12.2576213.short

The world’s largest digital camera is ready to investigate the dark universe | Space. (n.d.). Retrieved May 7, 2024, from https://www.space.com/dark-matter-lsst-camera-rubin-observatory

Thomas, S. J., Barr, J., Callahan, S., Clements, A. W., Daruich, F., Fabrega, J., Ingraham, P., Gressler, W., Munoz, F., Neill, D., Ribeiro, T., Sebag, J., Serrano, E., Stalder, B., Tighe, R., Vucina, T., & Xin, B. (2020). Vera C. Rubin Observatory: Telescope and site status. Ground-Based and Airborne Telescopes VIII, 11445, 68–82. https://doi.org/10.1117/12.2561581

University, M. L. D., Carnegie Mellon. (2022, August 19). Galaxies on Graph Neural Networks. Machine Learning Blog | ML@CMU | Carnegie Mellon University. https://blog.ml.cmu.edu/2022/08/19/galaxies-on-graph-neural-networks/

Vera C. Rubin Observatory. (2024). In Wikipedia. https://en.wikipedia.org/w/index.php?title=Vera_C._Rubin_Observatory&oldid=1220315709

Vilardell, F., Artigues, G., Sanz, J., García-Piquer, Á., Colomé, J., & Ribas, I. (2016). Using Robotic Operating System (ROS) to control autonomous observatories. Software and Cyberinfrastructure for Astronomy IV, 9913, 1065–1071. https://doi.org/10.1117/12.2232694

Voetberg, M., & Nord, B. (2023). DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling. DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling. DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling. https://doi.org/10.2172/2246791

Waldmann, I. P., & Griffith, C. A. (2019). Mapping Saturn using deep learning. Nature Astronomy, 3(7), 620–625. https://doi.org/10.1038/s41550-019-0753-8

Wan, A., Dunlap, L., Ho, D., Yin, J., Lee, S., Jin, H., Petryk, S., Bargal, S. A., & Gonzalez, J. (2020). NBDT: Neural-Backed Decision Trees. ArXiv. https://www.semanticscholar.org/paper/NBDT%3A-Neural-Backed-Decision-Trees-Wan-Dunlap/bbe4b35189bb94844ef394032522f82988874822

Watch: The scale of the entire Universe versus JWST’s views. (2023, February 20). Big Think. https://bigthink.com/starts-with-a-bang/jwst-universe/

Weiland, K. J. (2021). Future Model-Based Systems Engineering Vision and Strategy Bridge for NASA.

Welcome | Vera C. Rubin Observatory Project. (n.d.). Retrieved May 7, 2024, from https://project.lsst.org/

Yang, Y., Morillo, I. G., & Hospedales, T. M. (2018). Deep Neural Decision Trees. ArXiv. https://www.semanticscholar.org/paper/Deep-Neural-Decision-Trees-Yang-Morillo/a5d234d01da1cba10f8c28c65dc0ee42a6005220

Yin, J. E., Eisenstein, D. J., Finkbeiner, D. P., Stubbs, C. W., & Wang, Y. (2021). Active Optical Control with Machine Learning: A Proof of Concept for the Vera C. Rubin Observatory. The Astronomical Journal, 161(5), 216. https://doi.org/10.3847/1538-3881/abe9b9

Zucker, C., Goodman, A. A., Alves, J., Bialy, S., Foley, M., Speagle, J. S., Groβschedl, J., Finkbeiner, D. P., Burkert, A., Khimey, D., & Swiggum, C. (2022). Star formation near the Sun is driven by expansion of the Local Bubble. Nature, 601(7893), 334–337. https://doi.org/10.1038/s41586-021-04286-5

Abstract

This project explores the Kepler dataset to construct a predictive model of exoplanetary characteristics focusing on equilibrium temperature. We probed the mechanics between planets around their stars by adopting a multi-tiered approach encompassing Kline-McClintock Uncertainty Propagation, Monte Carlo simulations, and sensitivity analysis. Verification and validation processes quantified uncertainties, revealing the balance between orbital periods and impact parameters in determining planetary temperatures. Despite the inherent complexities and unpredictable nature of astrophysical systems, the project’s methodology, grounded in statistical analysis and systems engineering, gives the pathways of exoplanets newfound clarity. This study shows that integrating computational with astrophysical insight advances our understanding of the universe.

Introduction

In modern astrophysics, planets orbiting stars beyond our solar system have the potential to understand the universe. The Kepler telescope exoplanet survey stands at the center of this exploration. It offers an extensive database of known exoplanets (Borucki). Each is cataloged with detailed information about their characteristics and the methods used for their discovery. This data provides a window into our galaxy’s diverse and abundant planetary systems. The exoplanet survey is crucial in chronicling these discoveries as we shape our understanding of the universe and our place within it.

The survey’s significance is a vast collection of exoplanetary data and its role in ongoing research and exploration. The latest update lists thousands of confirmed exoplanets, which continues to grow thanks to the tireless efforts of missions and astronomers worldwide. These discoveries have profoundly shifted our perspective, revealing a universe teeming with planets as diverse as they are numerous. From gas giants larger than Jupiter to rocky worlds that might harbor conditions suitable for life, the variety of exoplanets challenges our understanding of planetary formation and evolution, sparking new questions and driving the search for answers (Exploring exoplanet populations with NASA’s Kepler Mission | PNAS).

Background

The Kepler Space Observatory, launched by NASA in 2009, represents a pivotal chapter in the annals of space exploration and the search for exoplanets. This mission was designed with a visionary goal: to explore star systems beyond our own for exoplanets, potentially unveiling other habitable worlds akin to Earth. Groundbreaking discoveries marked Kepler’s primary mission but faced mechanical challenges, leading to its original mission ending in 2013. Despite these setbacks, the telescope’s journey did not end there. In 2014, Kepler embarked on its “K2” extended mission, continuing its quest to uncover the secrets of distant planets and solar systems. This resilience has allowed Kepler to remain at the vanguard of exoplanetary research, consistently contributing invaluable data to the scientific community.
By May 2016, Kepler had verified 1,284 new exoplanets, significantly contributing to our understanding of the universe’s vast expanse. As of October 2017, the count of confirmed exoplanets discovered through various methods, including Kepler’s observations and ground-based efforts, had exceeded 3,000. This milestone underscores planets’ sheer abundance and diversity beyond our solar system, challenging our understanding of planetary formation and the conditions that might harbor life. Kepler’s discoveries have expanded our cosmic map and deepened our appreciation for the potential variety of planetary environments and compositions.

Our project is built upon this rich foundation laid by the Kepler Space Observatory. Utilizing the extensive dataset provided by Kepler, we aim to analyze further and classify these exoplanets, focusing on their potential habitability and characteristics. The data Kepler collected during its original and extended missions offers an unprecedented opportunity to study exoplanets in detail. We aim to harness advanced computational techniques and simulations, including Monte Carlo methods and machine learning algorithms, to extract meaningful insights from this data. By doing so, we hope to contribute to the growing body of knowledge about exoplanets and to continue the legacy of discovery that Kepler has so remarkably fostered.

Scope

Our project aims to understand exoplanet characteristics and classifications through advanced simulation techniques. Specifically, utilize the Kepler Exoplanet Search Results dataset to classify exoplanets accurately based on their observed factors, leveraging machine learning and statistical models (Shabram et al., 2020). Implement uncertainty quantification methods to assess and communicate the confidence and limitations of the classification models and predictions. Moreover, it demonstrates the effectiveness of simulation models in predicting exoplanetary attributes and their potential habitability based on the available data. It provides insights valuable for future astronomical research, especially in understanding the distribution and nature of exoplanets in the galaxy. This project’s challenges are ensuring the simulation models’ accuracy and reliability in classifying and predicting exoplanet characteristics. Nevertheless, this can employ cross-validation and other statistical techniques to fine-tune models and verify their performance. Another challenge we can encounter is effectively quantifying and interpreting the uncertainties inherent in astronomical data and model predictions. However, this can be solved by applying advanced techniques like Monte Carlo simulations and probabilistic models to quantify uncertainty and validate the models’ predictions. Another challenge is cohesively integrating concepts from astrophysics, data science, and machine learning. We will ensure that interdisciplinary resources and expertise are utilized effectively and a comprehensive approach combining these diverse fields. By addressing these challenges, the scope of the exoplanet simulation project is to advance our understanding of exoplanets and demonstrate the power and potential of simulation models in astronomical studies.

Conceptual and Computational Model

Our project bridges the gap between astronomical datasets and the intricate realities of exoplanets. This endeavor is structured around a comprehensive conceptual model that blends advanced astrophysical knowledge with data analysis techniques. We classify exoplanets based on their potential habitability, considering orbital characteristics, planetary properties, and stellar influences. This conceptual framework is translated into a computational model, utilizing MATLAB’s analytical capabilities. Here, we employ surrogate modeling, machine learning algorithms, and Monte Carlo simulations to predict and classify these entities and quantify and interpret the uncertainties inherent in such complex astronomical data. This synthesis of conceptual understanding and computational execution forms the core of our project (Changeat et al., 2020).

Conceptual Model

Enhanced Conceptual Model for Uncertainty Quantification in Exoplanet Classification.

Integration of Systems Engineering and Multidisciplinary Modeling

The conceptual model is grounded in the systems engineering approach, integrating multidisciplinary perspectives to understand the complex dynamics of exoplanets. This approach encompasses various aspects of astrophysics, data science, and machine learning, creating a framework for analyzing Kepler’s dataset. The model prioritizes accurately depicting exoplanetary systems, incorporating diverse data types such as orbital characteristics, planetary properties, and stellar influences. It emphasizes the need to understand individual exoplanets and their interaction within the broader context of their respective environment (Exploring

Exoplanets, 2023).

Emphasizing Model Verification, Validation, and Uncertainty Quantification

A core aspect of the conceptual model is verification and validation, ensuring the accuracy and reliability of the simulations used. This involves assessing the model’s fidelity in representing the real-world phenomena of exoplanetary systems and quantifying uncertainties inherent in observational data and model predictions. We employ probabilistic modeling and uncertainty analysis methods, such as Monte Carlo simulations, to gauge confidence in its predictions and understand the range of possible scenarios, particularly regarding exoplanet habitability and classification (Lucas et al., 2008).

Advanced Simulation Techniques for Predictive Analysis

The model extends to predictive simulations encompassing uncertainties. This includes surrogate modeling for approximating complex relationships, coupled modeling for system-level analysis, and sensitivity analysis for understanding the impact of various parameters. Using these techniques, the conceptual model aims to classify exoplanets based on their observed characteristics and explore potential future scenarios and outcomes, leveraging the rich dataset provided by the Kepler mission (Araujo, 2020).

Computational Model

We will develop a computational model focusing on using MATLAB for exoplanet classification.

MATLAB Implementation for Data Analysis and Simulation

The computational model translates the conceptual framework into executable simulations and analyses using MATLAB. We will use the Curve Fitting Toolbox and statistical functions, incorporating surrogate modeling techniques to simplify complex astrophysical relationships. Artificial neural networks, implemented through MATLAB’s Neural Network Toolbox, are used to classify exoplanets, capturing non-linear patterns and relationships within the Kepler data. The model employs MATLAB’s data processing capabilities to handle large datasets, facilitating efficient and accurate analysis (Akeson et al., 2013).

Model Validation and Uncertainty Analysis with Monte Carlo Simulations

A critical component of the computational model is the validation of the simulations against actual observational data and the comprehensive uncertainty analysis. MATLAB tools are used for cross-validation and performance assessment of the models. Monte Carlo simulations are implemented to explore the impact of uncertainties in the exoplanet data on the classification outcomes. This involves generating numerous scenarios by varying input parameters within their uncertainty bounds, thus providing a probabilistic understanding of the predictions (Bonavita et al., 2012).

Application of Multidisciplinary Optimization and Policy Analysis

The computational model also embraces multidisciplinary optimization techniques and policy analysis. This includes the application of discrete and dynamic simulations to model different aspects of exoplanetary systems and sensitivity analysis to determine how changes in model inputs affect outputs. The aim is to optimize the models for better accuracy and reliability in exoplanet classification, ensuring that the simulations can effectively inform policy and decision-making processes in astrophysics.

The conceptual and computational models leverage a comprehensive, multidisciplinary approach, integrating advanced simulation techniques, probabilistic modeling, and machine learning to analyze Kepler’s exoplanet data. This synthesis enhances our understanding of exoplanets and their environments, guided by systems engineering principles, model verification, and validation (Morton, 2012).

Exoplanets dataset

Based on the information from the provided documents, we can comprehensively understand the Kepler Objects of Interest (KOI) dataset, which is pivotal for our exploratory data analysis.

Kaggle repository

https://www.kaggle.com/datasets/nasa/kepler-exoplanet-search-results

Overview of the Dataset

The Kepler Objects of Interest dataset from the Kepler Space Observatory represents a rich and extensive collection of potential exoplanets. Each entry in this dataset is tagged with a unique identifier, the KOI number, which is crucial for tracking and research purposes. The dataset encompasses many parameters, from basic identification details like Kepler ID (kepid) and KOI Name (kepoi_name) to complex astrophysical measurements and calculations. These parameters provide in-depth insights into the characteristics of each exoplanet candidate, such as its orbital period, transit properties, and stellar parameters. This holistic dataset not only aids in identifying and classifying exoplanets but also enriches our understanding of their environments and potential habitability (Dataset — Kepler exoplanet project 0.1 documentation).

Key Parameters for Analysis

Several parameters within the dataset are particularly significant for our project. The ‘koi_disposition’ and ‘koi_pdisposition’ fields indicate the status of each exoplanet candidate, categorizing them into groups like CONFIRMED, CANDIDATE, or FALSE POSITIVE. This classification is vital for filtering genuine exoplanet candidates from false positives. The ‘koi_score’ provides confidence in the candidate’s status, aiding in the validation process. Parameters like ‘koi_period,’ ‘koi_impact,’ ‘koi_duration,’ and ‘koi_depth’ offer detailed insights into the orbital characteristics of the exoplanets. Furthermore, stellar parameters such as ‘koi_steff’ (stellar effective temperature), ‘koi_slogg’ (stellar surface gravity), and ‘koi_srad’ (stellar radius) contribute to understanding the host stars’ properties, which are essential for assessing the habitability potential of the orbiting exoplanets.

Dataset Application and Importance

The Kepler dataset’s comprehensive nature makes it an ideal resource for advanced astronomical research and data analysis. We can explore various aspects of exoplanetary science by employing statistical and machine learning techniques on this dataset. For instance, understanding the distribution and frequency of exoplanets within different stellar environments, assessing the likelihood of habitable conditions, and exploring the diversity of exoplanet types and sizes. Thus, The dataset serves as a foundational tool for our project, enabling us to bridge the gap between theoretical models and observational data to comprehend the mysteries of exoplanets.

Exploratory Data Analysis

The Kepler Space Telescope has provided an invaluable resource in the form of the Kepler Objects of Interest (KOI) dataset. This dataset is a compilation of potential exoplanets identified by the Kepler mission and our exploratory data analysis (EDA). Our EDA will apply statistical methods and visualization techniques to extract meaningful insights. By analyzing various parameters such as planetary dispositions, orbital characteristics, and stellar properties, we seek to deepen our understanding of the exoplanets. The Kepler dataset offers an opportunity to apply data science techniques in an astrophysical context. Histograms, boxplots, and scatter plots will be used to visualize and interpret the data, revealing underlying patterns and distributions. Fields like koi_disposition and koi_pdisposition provide a glimpse into the confirmation status of these exoplanet candidates, while koi_score offers a quantifiable measure of confidence in these classifications. Other parameters, such as orbital periods, planet radii, and stellar characteristics, are key to understanding these exoplanets’ physical attributes and environments. We will analyze these exoplanets effectively and explore several fields through this EDA.

Figure 1: During 9.6 years in orbit, Kepler led to the discovery of more than 2,600 planets by observing more than half a million stars. Image: NASA

Our project aims to use real observational data to measure model error and perform verification, validation, and predictive simulation inclusive of uncertainty. Selecting the most appropriate fields from the Kepler dataset is crucial. The fields we choose are relevant to our research objectives and contain enough information to enable meaningful analysis and modeling. For full field descriptions, refer to Appendix A. Based on the techniques we plan to use and the goals of our project, the following fields from the Kepler dataset would be most suitable:

koi_disposition

This field categorizes each KOI (Kepler Object of Interest) into various dispositions like CONFIRMED, CANDIDATE, or FALSE POSITIVE. It is essential for verifying and validating our model, providing a benchmark for comparing predicted classifications against actual outcomes.

koi_pdisposition

Similar to koi_disposition, this field indicates the disposition of the KOI based on Kepler data analysis. It helps understand the initial classification and assess model performance against these preliminary categorizations.

koi_score

This confidence score is associated with the KOI disposition. It can be used for uncertainty quantification and to gauge the reliability of the classifications in our predictive models.

koi_period, koi_time0bk, koi_impact, koi_duration, koi_depth

These fields provide information about the orbital characteristics of the exoplanets, which are critical for modeling their dynamics and interactions. They are valuable for surrogate modeling and sensitivity analysis.

koi_prad, koi_teq, koi_insol

These parameters describe the physical properties of the exoplanets, such as radius, equilibrium temperature, and incident stellar flux. They are crucial for assessing habitability and for multidisciplinary modeling.

koi_steff, koi_slogg, koi_srad

Stellar parameters are key to understanding the environment in which these exoplanets exist. They can influence the modeling of planetary systems and are essential for coupled modeling and system thinking.

koi_fpflag_nt, koi_fpflag_ss, koi_fpflag_co, koi_fpflag_ec

These flags indicate why a KOI might be classified as a false positive. Analyzing these fields can help refine the model and reduce prediction errors.

For predictive simulations and policy and sensitivity analysis, fields related to the orbital and physical properties of the exoplanets and their host stars will provide a comprehensive dataset to work with. Additionally, incorporating the disposition and confidence score allows for an approach to model verification and validation, enhancing the reliability of our research outcomes.

Visualization

Summary of KOI Scores:

Histogram of Koi Orbital Period

KOI Score Distribution by Disposition

Count of KOI Dispositions

Scatter Plot of KOI Planetary Radius vs Equilibrium Temperature

Equilibrium Temperature vs Planetary Radius Scatter Plot

Bar Plot of False Positive Flags

Verification Procedures & Uncertainty Quantification

Each technique contributes to the verification process by ensuring that our model accurately and reliably represents the underlying phenomena and that the uncertainties in model predictions are adequately quantified (Rider and Kamm, 2019).

The Modelling Process and Systems Engineering Approach

We use a systematic approach to verify our model’s integrity. This involves checking each step of the modeling process, from data preprocessing to the final output. Specifically, for fields like koi_period, we verify that our data transformation and cleaning procedures preserve the data’s integrity. We will compute the mean and standard deviation for the koi_period. We will check if the mean and standard deviation fall within an acceptable tolerance. We will focus on ensuring data integrity by checking that the key statistical properties of the data remain consistent after each processing step. Before preprocessing, we will compute the mean and standard deviation for the koi_period field. Then, we will remove the outliers. This step should be replaced with actual data preprocessing relevant to our project. Then, we will compute the mean and standard deviation for koi_period after preprocessing. Ultimately, we will see if the mean and standard deviation changes fall within an acceptable tolerance. This ensures that preprocessing steps have kept the statistical properties of the data the same.

The mean and standard deviation of the koi_period field before and after preprocessing remained consistent. This indicates that our preprocessing steps, such as outlier removal or data cleaning, did not significantly alter the statistical properties of the data. The fact that the mean and standard deviation remain unchanged (75.6714 and 1334.744, respectively) strongly indicates data integrity. This implies that the preprocessing steps retain the true nature of the astronomical observations. The assurance that preprocessing has not skewed the data allows us to proceed with further modeling and analysis. This is a key component in using the Kepler dataset for predictive simulations inclusive of uncertainty.

Probability Modeling

Probability modeling involves verifying the assumptions of our statistical models. For instance, if we assume a normal distribution for koi_teq, We will use the Kolmogorov-Smirnov Goodness of Fit Test (K-S test) to compare our data with a known distribution and let us know if our data have the same distribution (Arsenault, 2020).

Verification Hyphotesis:

P is our sample's distribution, and P0 is a specified distribution.

We verify the assumption of a normal distribution for a field koi_teq. We will use the Kolmogorov-Smirnov (KS) test, which compares the empirical distribution of our data with a specified theoretical distribution, in this case, the normal distribution. First, we clean the koi_teq data and check if it is standardized by subtracting the mean and dividing by the standard deviation. We perform the KS test. It returns a test decision (h) and a p-value (p). The test decision is 0 if the test fails to reject the null hypothesis that the data follows the specified distribution and 1 otherwise.
The graph shows the probability density of the koi_teq values overlaid with a normal distribution curve for comparison. The results of the Kolmogorov-Smirnov Goodness of Fit Test histogram display the empirical distribution of koi_teq values, showcasing how often each range of values occurs within the dataset. The superimposed red curve represents the normal distribution, the expected distribution if koi_teq follows a normal pattern. Visually, we observe a discrepancy between the histogram and the normal distribution curve (red line), especially in the distribution’s tails, indicating that koi_teq does not perfectly follow a normal distribution.

The K-S test quantitatively assesses the similarity between the observed and theoretical normal distributions. The test’s extremely small p-value (1.8632e-157) suggests that the differences between the observed koi_teq and the normal distribution are statistically significant. This means we reject the null hypothesis that koi_teq follows a normal distribution. The rejection of this assumption implies that models based on normal distribution assumptions may not be appropriate for koi_teq. This could affect the accuracy of predictions and the validity of inferences made from such models.

koi_teq does not follow a normal distribution (p-value = 1.8632e-157).

Surrogate Modeling

Verification ensures that the surrogate model accurately represents the more complex reality in surrogate modeling. We verify this by comparing the surrogate model’s predictions against those from a detailed physical model or actual observations (Graham-Brady, 2018).

Surrogate Model Verification:

Where Yi are observations, and Yi are surrogate model predictions.

The verification for the surrogate model will be performed by comparing the predictions from the surrogate model with those from a more detailed model. We will assume we have a surrogate model for predicting a feature like koi_teq. We will compare the predictions of this surrogate model with actual observations for verification. Our surrogate model will be a linear regression model. We will calculate the mean absolute error between the predictions and actual observation. This will provide a quantitative measure of the surrogate model’s accuracy.

The linear regression model has an intercept of approximately 1085.4 and a slope of approximately -0.00034754. This indicates that for every unit increase in the independent, the dependent variable koi_teq by 0.00034754 units on average, holding all else constant. The R-squared value is low (1.56e-06), which suggests that the model does not explain the variability in the data well. The adjusted R-squared is even negative, which indicates that the model is no better than a horizontal line. The F-statistic against a constant model is not significant (p-value = 0.905), implying that there is no linear relationship between the variables of interest as modeled.

The graph plots actual observations against the predictions made by the surrogate model. if the surrogate model were perfect, all points would lie on the dashed line representing perfect predictions. However, the scatter plot shows that the actual values deviate significantly from the predicted values, which are tightly clustered around the intercept value from the regression. This is consistent with the slope being close to zero, indicating that the model does not capture any trend or relationship. In this case, the linear regression is inadequate for predicting the feature of interest. It should either be refined or replaced with a more complex model that can account for the nonlinearity and variability in the data.

Coupled Modeling and System Thinking

Verification in coupled modeling ensures that interaction between different model components is correctly represented. For example, we are verifying that changes in koi_steff appropriately influence koi_srad in the model. We verify the interactions between different model components. The relationship between koi_steff , stellar effective temperature, and koi_srad, stellar radius, by the correlation between these two variables is compared with theoretical or empirical expectations. This measures the strength and direction of the linear relationship between these two variables.

The correlation between koi_steff and koi_srad within is negative -0.11807, suggesting a weak inverse relationship counterintuitive to the expected positive correlation. More important is that the correlation value is close to zero, indicating a very weak linear relationship. This could imply that the model may not accurately capture the expected physical relationship or that the Kepler data encompasses various stellar types and evolutionary stages, diluting the expected correlation. This discrepancy underscores the complexity of the data and suggests that further refinement of the model is necessary, including stratification by star type and life cycle stage. It also raises questions about whether additional variables or non-linear modeling techniques should be incorporated to better capture the physical realities of stellar characteristics.

Expected Correlation derived from the dataset -0.11807

Applied Discrete and Dynamic Simulations

We verify that the model behaves as expected over time in discrete and dynamic simulations. This can be done by checking the stability and accuracy of the simulation outputs against known benchmarks or theoretical predictions. Based on a hypothetical model, we simulate koi_teq, equilibrium temperature, and change over time and then verify its stability. The final value of the simulated koi_teq is compared against the expected final value calculated based on the model. A small tolerance is allowed to account for numerical precision issues.

We examine the model’s behavior over time to ensure it aligns with expected patterns. The simulation presents a stable, linear increase in koi_teq over time. This indicates a model that consistently applies the same rules or formulas at each time step without showing erratic behavior or divergence from the expected trend. This suggests that the model is following the prescribed dynamics accurately. It is also essential to establish the boundary conditions and the theoretical underpinnings that would justify a linear progression of koi_teq over time to strengthen the verification process. The success of this verification step is foundational for trusting the model in predictive simulations where the behavior over time will have implications.

Simulation is stable and follows the expected pattern.

Policy and Sensitivity Analysis

In sensitivity analysis, we verify our model by assessing how input change in koi_impact affects the outputs. This helps in quantifying the uncertainty associated with model predictions.

Sensitivity Analysis Formula:

Where Sx is the sensitivity of the output f to the input variable x.
We will analyze this by observing how koi_impact, koi_impact, and koi_duration variations affect model output. We will perform sensitivity analysis by varying koi_impact and observing the changes in the predicted koi_teq.

The regression shows that koi_period and koi_duration have significant p-values, indicating that these predictors have statistically significant effects on the koi_teq. However, koi_impact has a p-value much greater than 0.05, suggesting that its effect is not statistically significant within the context of this model. The R-squared value is relatively low, which implies that the model explains only a small portion of the variability in the koi_teq. This might indicate that other factors not included in the model could be influencing the equilibrium temperature or that the relationship between these variables and koi_teq is non-linear or involves interactions that have not been accounted for. The sensitivity analysis visual demonstrates the influence of koi_impact on the predicted equilibrium temperature of the exoplanet koi_teq. The plot shows a clear trend indicating that as koi_impact increases, the predicted koi_teq decreases. Even though the p-value for koi_impact was insignificant in the regression model, the sensitivity analysis might still reveal a relationship when it is the only manipulated variable. This could suggest that koi_impact may have a more complex relationship with koi_teq that a linear model doesn’t capture or that its effect is only apparent when not considering the influence of other variables.

Monte Carlo Simulations

Monte Carlo simulations verify the impact of input variability on model outputs. By running the model numerous times with varied inputs(What is Monte Carlo Simulation), we can quantify the range and distribution of the outputs, thus verifying the model.

Monte Carlo Uncertainty Quantification:

Where Sigma MC is the standard deviation of the Monte Carlo outcomes. We will conduct a Monte Carlo simulation to understand how variations in koi_impact and koi_period affect koi_teq. We will use the linear regression model to predict koi_teq based on koi_impact and koi_period. The model predicts koi_teq for each set of randomized inputs, and the results are stored in an array. By examining the spread and shape of the output distribution, we can understand the model and the uncertainty associated with its predictions.

The linear regression model coefficients indicate the relationship between koi_teq and the inputs (koi_period and koi_impact). The negative coefficients suggest that as koi_period and koi_impact increase, koi_teq tends to decrease. This relationship, along with the R-squared value, which is quite low, suggests that the model explains only a small portion of the variability in koi_teq, indicating potential room for model improvement.

The histogram shows a roughly symmetrical distribution of the simulated koi_teq values around the central peak. However, bars at the extreme ends (-3000 and 1000) suggest some cases where the model predictions are significantly different from the mean prediction. The central part of the histogram, where the bars are taller, represents the most frequent outcomes from the simulation. The data around this region is where the koi_teq predictions lie, given the current model and the variability in koi_impact and koi_period. The spread of the histogram, including the tails, provides an insight into the uncertainty of the model’s predictions. If the spread is wide with substantial frequency at the tails, it suggests a higher uncertainty. Conversely, the model has lower predictive uncertainty if most outcomes are concentrated around the mean.

Verification Conclusion

The verification process revealed both expected and unexpected patterns within the Kepler dataset. For instance, the mean and standard deviation of the koi_period remained consistent pre and post-processing, confirming the preservation of the data’s integrity through various preprocessing steps. The application of the Kolmogorov-Smirnov test highlighted a significant deviation from the normal distribution for koi_teq, suggesting that the underlying data may not conform to standard statistical models. The surrogate model’s predictions were quantitatively assessed against actual observations, indicating a notable divergence. The weak correlation between koi_steff and koi_srad challenges preconceived notions about the direct relationship between these variables, suggesting a more intricate interplay influenced by various stellar characteristics. The verification process shows the evaluation of the importance of rigorous statistical assessments in predictive simulation. It has deepened the understanding of the system’s complexity and highlighted the value of embracing uncertainty as an integral part of modeling and simulation in astrophysics.

Validation Procedures & Uncertainty Quantification

Validation procedures, including uncertainty quantification, require a detailed model performance analysis against real-world data. For the Kepler dataset, validation involves assessing how well our model predictions align with known exoplanet characteristics. Each validation technique builds a reliable Kepler dataset analysis model. The aim is to ensure that the model’s predictions are theoretically sound and practical.

Introduction to Modeling for Experiments

In this phase, we compare model predictions with observational data.

Formula:

Where Oi is the observed value, Pi is the predicted value from the model, and N is the number of data points.

The validation results indicate how well the linear regression model fits the actual observational data. In this case, the model predicts koi_teq using koi_period as the predictor. The coefficient for koi_period is -0.023869, which indicates a small negative relationship between koi_period and koi_teq. As the orbital period increases, the equilibrium temperature koi_teq slightly decreases. The coefficient is statistically significant, given that the p-value is well below the 0.05 threshold. The R-squared value of 0.00204 is very low, which suggests that koi_period alone does not explain much of the variability in koi_teq. The adjusted R-squared, which accounts for the number of predictors in the model, is also low, reinforcing that koi_period is not a strong predictor of koi_teq. The Root Mean Squared Error (RMSE) of 858 and the Mean Squared Error (MSE) of 721871.7979 are measures of the model’s predictive error. They indicate that, on average, the model’s predictions of koi_teq deviate from the actual observed values by a considerable margin. The model’s predictive power is limited, given the low R-squared value and the high RMSE and MSE. This suggests that other factors not included in the model might influence koi_teq and should be considered for a more accurate prediction.

Mean Squared Error: 721871.7979

Design of Validation Experiments

Design experiments to compare the model’s predictions with actual Kepler observations, focusing on specific fields like koi_period and koi_impact. We conduct a regression analysis and calculate the coefficient of determination (R²). The R² statistic measures how well the model replicated the observed outcomes based on the proportion of total variation of outcomes explained by the model.

Formula:

Where O dash is the mean of observed values.

The linear regression model used in the validation process includes koi_period and koi_impact as independent variables to predict koi_teq. The estimated coefficients indicate that both koi_period and koi_impact have significant effects on koi_teq, with the orbital period having a negative coefficient, suggesting that a longer orbital period is associated with a lower equilibrium temperature. This relationship is consistent with theoretical expectations since objects with a longer orbital period are generally further from the star and thus receive less stellar radiation, leading to lower temperatures.

The Mean Absolute Error (MAE) and the residual plots provide additional insight into the model’s accuracy and the distribution of errors. The MAE of 583.302407 suggests that, on average, the model’s predictions are approximately 583.3 units away from the actual observed koi_teq values. The residual plot shows the difference between the observed and predicted koi_teq values against the predicted koi_teq. However, the plot shows a noticeable pattern, especially at higher predicted koi_teq values, suggesting the presence of systematic errors in the model predictions. The R-squared value of approximately 0.002 indicates that the model explains very little of the variability in the observed koi_teq data. This could indicate that additional variables or more complex modeling techniques may be needed to capture the underlying relationships better.

Probability Modeling

Assess the probability distribution of the model’s residuals to ensure they are typically distributed, indicating a good fit. We can use the distribution of a model’s residuals to perform probability modeling as part of validation procedures. Specifically, we will check if the residuals follow a normal distribution. This can be done using a normality test, such as the Shapiro-Wilk test, or visually with a histogram and a Q-Q (quantile-quantile) plot. We will continue with the model predicting koi_teq based on koi_period and koi_impact.

Formula:

Where mu are the mean and sigma standard deviation of the residuals.

The model suggests a statistically significant negative relationship between koi_period and koi_teq, as indicated by a negative coefficient and a very small p-value (2.514e-06). This implies that the equilibrium temperature decreases as the orbital period increases. However, the relationship between koi_impact and koi_teq is not statistically significant, given its larger p-value (0.34963), indicating that within this model, koi_impact does not have a clear linear effect on koi_teq. The R-squared value is very low (0.00251), which means that only a very small portion of the variability in koi_teq is explained by koi_period and koi_impact in this model. The adjusted R-squared is also very low (0.00229), which considers the number of predictors in the model relative to the number of observations and suggests that the model needs to improve more upon the baseline model. Despite the low R-squared value, the F-statistic indicates that the model is statistically significant (p-value = 9.76e-06). This means that the predictors significantly affect the outcome when considering the model as a whole. The Kolmogorov-Smirnov test result (h) is 1, which means the test rejects the null hypothesis that the residuals come from a normally distributed population. The extremely small p-value (1.0056e-163) further confirms this result. This non-normal distribution of residuals is problematic for linear regression, which usually assumes distributed errors.

The probability modeling validation process results suggest that the residuals of the koi_teq predictions based on koi_period and koi_impact do not follow a normal distribution. This can be inferred from both the histogram and the Q-Q plot provided.

The histogram shows that the residuals are heavily skewed towards the right, showing a long tail that deviates from the expected bell curve of a normal distribution. The red curve representing a normal fit does not align well with the histogram bars.

The Q-Q plot shows an apparent deviation from the red reference line, especially at the tails. The residuals are not normally distributed. The results of the Kolmogorov-Smirnov test support these visual assessments.

Surrogate Modeling

Use surrogate models to replicate the complex relationships in the Kepler data and validate these models against subsets of the data. We can construct a surrogate using polynomial regression to approximate the relationships between various exoplanetary features. We will validate this surrogate model against a subset of the data. This surrogate model serves to simplify the complex relationships in the Kepler data and is validated using MSE, a measure of the model’s prediction error. A lower MSE indicates a better model fit.

Formula:

A surrogate model can be a polynomial regression, such as

The polynomial regression surrogate model constructed attempts to capture the nonlinear relationships between koi_period, koi_impact, and the equilibrium temperature koi_teq. The coefficients from the regression model give us insight into how each term contributes to predicting koi_teq. A negative coefficient for koi_period suggests that the equilibrium temperature decreases as the orbital period increases. The positive coefficient for koi_impact indicates that higher impact values are associated with higher equilibrium temperatures. However, the p-value for koi_impact is above the typical alpha level of 0.05, indicating that the relationship may not be statistically significant. Though not high, the R-squared value of 0.154 indicates that the model explains approximately 15.4% of the variance in koi_teq, which is reasonable given the complexity and variability of exoplanetary characteristics. The Mean Squared Error (MSE) measures 600166.4846, suggesting variability in the model’s predictive accuracy, which is expected in complex systems such as exoplanet observation.

The 3D plot showing the surrogate model validation visually compares observed and predicted koi_teq. Ideally, the observed and predicted points should align closely. Deviations indicate discrepancies between the model’s predictions and actual observations, highlighting areas where the model may be improved.

Multidisciplinary Modeling and Optimization

Integrate various modeling techniques, such as statistical models and machine learning algorithms, and validate their combined predictive power. We will use koi_period, koi_impact, and koi_duration as predictors to estimate koi_teq. First, we create a linear regression model and a decision tree model, then combine their predictions and assess the combined model’s performance. The model captures both linear and non-linear relationships in the data. A lower MSE indicates a better fit of the combined model to the data.

The linear regression model, which predicts the equilibrium temperature (koi_teq) based on the orbital period (koi_period), impact parameter (koi_impact), and transit duration (koi_duration), presents a statistically significant relationship between the predictors and the outcome variable. This is evident from the t-statistics and p-values associated with the predictors, particularly koi_period and koi_duration, indicating that these variables significantly affect the predicted koi_teq. The negative coefficient for koi_period suggests an inverse relationship with koi_teq, while the large negative coefficient for koi_duration indicates a substantial decrease in koi_teq with longer transit durations. The Root Mean Squared Error (RMSE) of 839 for the linear regression model signifies the standard deviation of the residuals, which are the prediction errors. While the R-squared value of 0.0392 is relatively low, indicating that only a small percentage of the variance in koi_teq is explained by the model, this is not uncommon in complex systems such as exoplanet characteristics, where many unmodeled factors may influence the outcomes. The Combined Model Mean Squared Error (MSE) of 459736.5778 provides insight into the average squared difference between the observed and predicted koi_teq. Although the MSE is relatively high, it is crucial to consider it within the context of the data’s variability and the scale of koi_teq values.

Combined Model Mean Squared Error (MSE): 459736.5778

Coupled Modeling and System Thinking

Ensure the interactions between different model components are accurately reflected and validated against known astrophysical relationships. This process involves validating that changes in koi_slogg, the surface gravity log, appropriately influence another koi_srad, the stellar radius, by known astrophysical relationships. We will use a linear regression model using koi_slogg to predict koi_srad. Then, validate this model against known astrophysical relationships. In astrophysics, a known relationship exists between a star’s surface gravity and radius, which we can use as a benchmark for validation. The MSE measures the model’s accuracy, and the coefficients give insights into how well the model captures the relationship between koi_slogg and koi_srad.
The validation results for the coupled modeling approach demonstrate a significant relationship between the star’s surface gravity (koi_slogg) and its radius (koi_srad). The negative coefficient for koi_slogg suggests that as the surface gravity increases, the stellar radius decreases, which is consistent with astrophysical principles that a more positive intercept (44.227) indicates the expected value of koi_srad when koi_slogg is zero. This coefficient (-9.8536) is particularly significant, as indicated by its large t-statistic and p-value of zero, meaning there is a strong inverse relationship between surface gravity and stellar radius within the Kepler dataset. This value indicates that approximately 41.3% of the variability in the stellar radius is explained by the model, which is a moderate amount of explanatory power. Root Mean Squared Error (RMSE) (5.14) & and Mean Squared Error (MSE) (12.759) provide a measure of the average error magnitude in the radius predictions. The relatively low RMSE and MSE values indicate that the model predictions are, on average, close to the observed values. Thus the model has a reasonable level of accuracy.

Model Coefficients: 44.2273 -9.8536

Applied Discrete and Dynamic Simulations

Validate the time-dependent simulations against known temporal patterns or theoretical predictions in exoplanet behavior. We consider variables that change over time, such as the koi_period orbital period of the exoplanet, and validate our simulations against expected astrophysical behaviors or theoretical models. We will create a time-series model that predicts changes in the koi_period based on other relevant time-variant variables. The model’s coefficients provide insights into the nature of the temporal relationship captured by the model.

The model shows a statistically significant temporal link, but its predictive power is low, suggesting that exoplanet orbital period dynamics require more complex modeling with additional predictive elements or non-linear correlations.

The time-series model used to predict the exoplanet’s orbital period, koi_period, represents time as koi_time0bk, the first transit in Barycentric Kepler. The regression model coefficients show a strong relationship between time and orbital period, as seen by the negative intercept and positive slope for koi_time0bk.

The large F-statistic (p-value = 7.43e-21) appears resilient. R-squared is low (0.00914), indicating that the model explains little koi_period variation. Time may not explain exoplanet orbital period variety despite a statistically significant connection.

Due to its 1.33e+03 Root Mean Squared Error (RMSE), the model’s predictions may be inaccurate. The Mean Squared Error (MSE) of 721871.7979, the average squared difference between observed actual occurrences and model projections, also shows significant prediction error.

Policy and Sensitivity Analysis

Conduct sensitivity analysis to understand how changes in input koi_period affect model predictions and validate these effects against expected trends. Let us use a linear regression model predicting koi_teq, equilibrium temperature, based on koi_period and koi_impact. We will perform sensitivity analysis on koi_period and validate the results. Vary koi_period within a reasonable range, keeping koi_impact constant at its mean value. For each koi_period value, predict koi_teq using the trained model. By comparing the observed trend in the sensitivity analysis with expected theoretical models, we validate whether the model’s response to changes in koi_period aligns with known behaviors.

From the results, we can infer a negative slope, which means that as the orbital period (koi_period) increases, the equilibrium temperature (koi_teq) decreases. This is consistent with the negative coefficient for koi_period in the regression model. The coefficient for koi_period is significant (p-value = 2.514e-06), which confirms that koi_period is an influential predictor for koi_teq in our model. On the other hand, koi_impact has a p-value of 0.34963, suggesting it is not a significant predictor in the presence of koi_period. The R-squared value is very low (0.00251), indicating that the model does not explain much of the variability in the koi_teq. This suggests that other factors not included in the model may influence koi_teq. The Root Mean Squared Error (RMSE) of 855 is relatively high, which could indicate that the model predictions are, on average 855 units away from the actual koi_teq values. The plots predicted equilibrium temperature (koi_teq) against the orbital period (koi_period). The graph shows that the highest temperatures are associated with lower orbital periods.

Monte Carlo Simulations

Run the model multiple times with varying inputs to assess the range of predictions and validate these against the observed variability in exoplanet characteristics. We perform Monte Carlo simulations by varying key input parameters and observing the effects on the model’s predictions. We will use predict koi_teq, equilibrium temperature, using koi_period and koi_impact as inputs. We will generate multiple sets of input data by random sampling within the range of koi_period and koi_impact. Then, through the model, observe the distribution of the predicted koi_teq. The distribution of the predicted values will be compared with observed variability in exoplanet characteristics to validate the model’s performance.

The linear regression model’s coefficients suggest that koi_period has a statistically significant negative effect on koi_teq, as indicated by the negative t-stat and low p-value, while koi_impact’s effect is not statistically significant (high p-value). However, the R-squared value is extremely low, suggesting that the model explains very little of the variability in the data, which is consistent with the widespread predictions in the Monte Carlo simulation. The histogram shows the frequency distribution of predicted equilibrium temperatures (koi_teq) from the Monte Carlo simulations. The x-axis represents the predicted equilibrium temperature, while the y-axis shows the frequency of these predictions occurring within the simulation runs. The distribution appears to be reasonably symmetrical around a central value close to zero, indicating that the model’s predictions for koi_teq, while varying due to the stochastic nature of the input parameters (koi_period and koi_impact), do not show a systematic bias towards overestimating or underestimating the equilibrium temperature. This symmetry is a good sign as it suggests the model is not systematically skewed.

Validation Conclusion

The validation process applied to the Kepler dataset reveals that the model’s predictability for exoplanet characteristics such as equilibrium temperature (koi_teq) based on orbital period (koi_period) and impact (koi_impact) is statistically significant yet limited in explanatory power. The factual evidence suggests that while specific inputs are influential, they do not solely account for the observed outcomes. The significant deviation of residuals from a normal distribution suggests a reconsideration of standard statistical assumptions, pointing towards more advanced or non-linear modeling techniques to capture the underlying patterns in the data. Through sensitivity analysis and Monte Carlo simulations, the validation process systematically quantifies the uncertainty and validates the model. These procedures demonstrate the predictive model’s sensitivity to input variations and its performance in reflecting the stochastic nature of the underlying physical processes. The validation process has provided a critical assessment of the model’s performance, revealing areas for improvement and affirming the need for a dynamic modeling framework that embraces the inherent uncertainty in astrophysical systems.

Predictive Simulation Inclusive of Uncertainty

To perform a predictive simulation inclusive of uncertainty, we will integrate various modeling techniques and uncertainty quantification methods to ensure reliability in our predictions. We will take a systematic approach to handle different aspects of modeling and uncertainty (The Uncertainty of Detecting Planets, 2019).

Kline-McClintock Uncertainty Propagation

Kline-McClintock’s method is key for our uncertainty analysis simulations. It will quantify the effect of measurement errors on output predictions (Kline-Mcclintock Method of Experimental Uncertainty). This method is particularly effective when dealing with linear or near-linear systems. It is suitable for our Kepler dataset analysis, where input uncertainties in fields like koi_period and koi_impact could significantly impact the predicted koi_teq.

The Intercept (1089.5 is the expected value of koi_teq when both koi_period and koi_impact are zero. It serves as a baseline in the model. koi_period Coefficient (-0.030872) suggests a negative correlation between the orbital period and the equilibrium temperature. As the orbital period increases, the equilibrium temperature will decrease slightly. koi_impact Coefficient (-2.4909 indicates a negative relationship, but given the more significant coefficient (in absolute terms) compared to koi_period, changes in koi_impact might have a more substantial effect on koi_teq. The p-value for koi_period is significantly small (2.514e-06), indicating that the relationship between koi_period and koi_teq is statistically significant. On the other hand, the p-value for koi_impact is large (0.34963), suggesting that koi_impact may not be a significant predictor of koi_teq in this model. The narrowness of the bars in the histogram indicates that the output uncertainty is very concentrated around a specific value, suggesting that the input uncertainties might not lead to a wide variation in the predicted koi_teq. This is a sign of a strong model where small input variations do not cause large changes in the output.

Monte Carlo Uncertainty Propagation

Monte Carlo simulations are invaluable for handling complex systems with numerous input variables (Bonavita et al., 2012). By running thousands of simulations with randomized input values within their expected ranges, we can obtain a distribution of the output, in this case, koi_teq. This approach allows us to visualize the impact of input uncertainties on our predictions, providing a robust way to understand the range of possible outcomes.

The CDF graph shows how the probability accumulates as the value of koi_teq increases. This indicates that the majority of the simulated values are less than 1100. The function is steep around 1000, suggesting that the values of koi_teq are densely packed around this range. The histogram displays the frequency distribution of the simulated koi_teq values. The normal distribution for the bell-shaped displayed is centered around a value slightly above 1000. The highest frequency of values is in the bin containing the median value, and the frequencies decrease as the values move away from this center. There is a slight asymmetry with a longer tail to the right, indicating a small number of simulations resulted in values of koi_teq higher than the central peak. The Monte Carlo simulation suggests that for this particular model, most of the koi_teq values will lie close to the mean with a predictable variance.

Confidence Interval

Confidence intervals are a statistical tool used to estimate the reliability of simulation results. By calculating these intervals, we can ascertain the range within which the true value of our predictions is likely to fall, providing a quantifiable measure of the model’s precision and reliability.

The confidence interval indicates that with 95% confidence, the true value of the parameter we are estimating temperature, koi_teq, from the Monte Carlo simulation, lies between 1002.674139 and 1171.431930. This range provides a measure of the precision of our model predictions. If the confidence interval is narrow, it suggests that our model predictions are precise; a wider interval indicates less precision and more uncertainty in the model’s predictions. This confidence interval provides a statistically valid range, which is critical for interpreting the results of our dataset, offering insights into the level of confidence we have in the simulation’s predictive capabilities.

95% Confidence Interval [1002.674139, 1171.431930]

Applying Model Bias Error

Accounting for known biases in the model enhances its predictive accuracy. By identifying and adjusting for these biases, we can correct the simulation outputs, bringing them closer to real-world values. This step is essential for models that may have inherent systematic errors.

The CDF graph represents the corrected equilibrium temperatures koi_teq after accounting for known model biases. The graph shows how the bias correction has shifted the distribution of koi_teq. This shift should make the simulated data more representative of the actual data. We can understand the likelihood of observing a temperature less than or equal to that value in our simulation for any given equilibrium temperature. This helps assess bias correction’s impact on the simulation outcomes’ overall distribution. This corrected CDF also provides a way to validate the model post-correction. The model’s reliability increases if the corrected outputs align well with empirical data or theoretical expectations.

Quantifying Sensitivity to Key Inputs

Sensitivity analysis helps us understand the impact of changes in input variables on the simulation outcomes. By systematically varying key inputs and observing the effects on the outputs, we can identify which inputs have the most significant influence on the model’s predictions.
The sensitivity analysis suggests that as the koi_period increases, the predicted equilibrium temperature decreases. This could imply that planets with longer orbital periods are expected to have lower equilibrium temperatures in the model. This makes intuitive sense if we consider that planets with longer orbital periods are generally farther away from their host stars, leading to lower temperatures due to less incident starlight. Similar to the koi_impact sensitivity analysis.

Cumulative Distribution Function Analysis

The CDF is a powerful tool for analyzing the distribution of simulation outputs. It provides insights into the probability distribution of the predictions, allowing us to understand the likelihood of different outcomes. This analysis is instrumental in assessing the range and distribution of possible results from our model.

The CDF for the sensitivity analysis on koi_period suggests that the line on the plot rises from a lower value of koi_period at -3000 to a higher value of 1000. A broad range of possible koi_teq values results from the variability in koi_period. By the end of the sensitivity analysis range, all possible koi_period values have been covered. If the CDF were to be used for decision-making or risk assessment, it could help determine the likelihood of different koi_period outcomes and thus aid in understanding the robustness of the predictions in relation to this input variable. The CDF for the sensitivity analysis on koi_impact suggests that it starts from a lower value of koi_impact, around 830, and rises to a higher value, around 1000. The coordinates (830, 0) and (1000, 1) indicate the range of koi_impact considered in the sensitivity analysis and the cumulative probability associated with these values. The start of the CDF at (830, 0) implies that the smallest koi_impact value included in the analysis is 830. The zero probability at this point means that none of the simulation runs resulted in a koi_impact less than this value. As the koi_impact increases, the CDF rises, indicating that the probability of encountering a koi_impact value less than or equal to a given value also increases. The end of the CDF at (1000, 1) means that the highest value in the sensitivity analysis for koi_impact is 1000, and all possible values have been included in the cumulative probability. This is where the CDF reaches the point where 100% of the simulation data is accounted for.

Simulation Conclusion

The simulations confirm the intricate relationship between koi_period and koi_teq, with a statistically significant inverse correlation and a less pronounced, though not statistically significant, relationship between koi_impact and koi_teq. The results underscore the substantial influence of koi_period on equilibrium temperatures, a fact well-grounded in astronomical theories. The simulations enhance our understanding of exoplanetary systems’ dynamics. The results from the Kline-McClintock Uncertainty Propagation and Monte Carlo simulations collectively paint a picture of these phenomena’ inherent variability and complexity. They suggest that while we can approximate behaviors and trends, absolute precision remains elusive due to the stochastic nature of the inputs and the multifaceted interactions within the systems. The simulation has highlighted the unpredictability inherent in modeling complex astrophysical systems and the importance of integrating various uncertainty quantification techniques to enhance the reliability of predictive models.

MATLAB Code

Exploratory Data Analysis

% Exploratory Data Analysis 
% Load the Kepler dataset
filePath = 'cumulative.csv';
keplerData = readtable(filePath);


% Histogram for 'koi_period'
figure;
histogram(keplerData.koi_period);
title('Histogram of KOI Orbital Period');
xlim([0 10^3]) %uncommented for the paper screenshot
xlabel('Orbital Period (days)');
ylabel('Frequency');


% Boxplot for 'koi_score'
figure;
boxplot(keplerData.koi_score, keplerData.koi_disposition);
title('Boxplot of KOI Scores by Disposition');
xlabel('Disposition');
ylabel('KOI Score');

% Count Plot for 'koi_disposition' (Category plot)
figure;
categories = categories(categorical(keplerData.koi_disposition));
counts = countcats(categorical(keplerData.koi_disposition));
bar(categories, counts);
title('Count of KOI Dispositions');
xlabel('Disposition');
ylabel('Count');

% Scatter Plot for 'koi_prad' vs 'koi_teq'
figure;
scatter(keplerData.koi_prad, keplerData.koi_teq, 'filled');
title('Scatter Plot of KOI Planetary Radius vs Equilibrium Temperature');
xlabel('Planetary Radius (Earth radii)');
ylabel('Equilibrium Temperature (K)');



% Multivariate Scatter Plot (e.g., 'koi_prad', 'koi_teq', and 'koi_insol')
figure;
gscatter(keplerData.koi_prad, keplerData.koi_teq, keplerData.koi_disposition);
title('Multivariate Scatter Plot');
xlabel('Planetary Radius (Earth radii)');
ylabel('Equilibrium Temperature (K)');
legend('Disposition');
%xlim([0 1]) %uncommented for the paper screenshot


% Bar Plot for False Positive Flags
figure;
fpflags = table(keplerData.koi_fpflag_nt, keplerData.koi_fpflag_ss, keplerData.koi_fpflag_co, keplerData.koi_fpflag_ec);
bar(sum(table2array(fpflags)));
set(gca, 'xticklabel',{'NotTransit','StellarEclipse','CentroidOffset','Contamination'});
xtickangle(45);
title('Bar Plot of False Positive Flags');
ylabel('Count');



% Summary of 'koi_score'
disp('Summary of KOI Scores:');
disp(['Mean KOI Score: ', num2str(nanmean(keplerData.koi_score))]);
disp(['Median KOI Score: ', num2str(nanmedian(keplerData.koi_score))]);
disp(['Standard Deviation of KOI Score: ', num2str(nanstd(keplerData.koi_score))]);

% Analyzing the relationship between 'koi_disposition' and 'koi_score'
% Calculate mean koi_score for each koi_disposition category
uniqueDispositions = unique(keplerData.koi_disposition);
for i = 1:length(uniqueDispositions)
    disposition = uniqueDispositions{i};
    meanScore = nanmean(keplerData.koi_score(strcmp(keplerData.koi_disposition, disposition)));
    disp(['Average KOI Score for ', disposition, ': ', num2str(meanScore)]);
end



% Convert 'koi_disposition' to a categorical variable
keplerData.koi_disposition = categorical(keplerData.koi_disposition);

% Histogram of 'koi_score' for different 'koi_disposition'
figure;
gscatter(keplerData.koi_score, keplerData.koi_disposition);
title('KOI Score Distribution by Disposition');
xlabel('KOI Score');
ylabel('Disposition');

Verification of Kepler’s dataset

% Verification Kepler's dataset
filePath = 'cumulative.csv';
keplerData = readtable(filePath);

% Modeling Process and Systems Engineering Approach
fprintf('\n<strong>Modeling Process and Systems Engineering Approach</strong>\n');
% Load the Kepler dataset

df = keplerData;
% Calculate original statistics of 'koi_period'
originalMean = mean(df.koi_period, 'omitnan');
originalStd = std(df.koi_period, 'omitnan');

% Display original statistics
disp(['Original Mean of koi_period: ', num2str(originalMean)]);
disp(['Original Standard Deviation of koi_period: ', num2str(originalStd)]);

% Data Preprocessing: Removing rows with missing 'koi_period' values
dfCleaned = rmmissing(df, 'DataVariables', {'koi_period'});

% Calculate statistics after cleaning
cleanedMean = mean(dfCleaned.koi_period, 'omitnan');
cleanedStd = std(dfCleaned.koi_period, 'omitnan');

% Display cleaned statistics
disp(['Cleaned Mean of koi_period: ', num2str(cleanedMean)]);
disp(['Cleaned Standard Deviation of koi_period: ', num2str(cleanedStd)]);

% Verify that the preprocessing steps have not significantly altered the data
tolerance = 0.05; % Define an acceptable tolerance level
if abs(originalMean - cleanedMean) / originalMean < tolerance && ...
   abs(originalStd - cleanedStd) / originalStd < tolerance
    disp('Data integrity verified after preprocessing.');
else
    disp('Data integrity issue detected after preprocessing.');
end

% Probability Modeling
fprintf('\n<strong>Probability Modeling</strong>\n');
% Ensure 'koi_teq' is available and without missing values
df = rmmissing(keplerData, 'DataVariables', 'koi_teq');

% Standardize 'koi_teq' data
standardizedData = (df.koi_teq - mean(df.koi_teq)) / std(df.koi_teq);

% Perform the Kolmogorov-Smirnov test against a normal distribution
[h, p] = kstest(standardizedData);

% Display the results
if h == 0
    disp(['koi_teq follows a normal distribution (p-value = ', num2str(p), ').']);
else
    disp(['koi_teq does not follow a normal distribution (p-value = ', num2str(p), ').']);
end


% Calculate mean and standard deviation of 'koi_teq'
mu = mean(df.koi_teq);
sigma = std(df.koi_teq);

% Create a histogram of 'koi_teq'
figure;
histogram(df.koi_teq, 'Normalization', 'pdf', 'DisplayName', 'Data Distribution');
hold on;

% Create a range of values for plotting the normal distribution
x = linspace(min(df.koi_teq), max(df.koi_teq), 100);

% Plot the normal distribution with the same mean and std as the data
normDist = normpdf(x, mu, sigma);
plot(x, normDist, 'r', 'LineWidth', 2, 'DisplayName', 'Normal Distribution');

% Labels and title
title('Distribution of koi_teq Compared to Normal Distribution');
xlabel('koi_teq');
ylabel('Probability Density');
legend;

hold off;


% Surrogate Modeling Verification
fprintf('\n<strong>Surrogate Modeling Verification</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_prad', 'koi_teq'});
X = df.koi_prad;
Y = df.koi_teq;

% Fit a linear regression model (Surrogate Model)
lm = fitlm(X, Y);

% Display the model
disp(lm);

% Predict 'koi_teq' using the surrogate model
predictedY = predict(lm, X);


% Calculate the prediction error
predictionError = Y - predictedY;

% Display the mean absolute error (MAE)
mae = mean(abs(predictionError));
disp(['Mean Absolute Error: ', num2str(mae)]);

% Plotting actual vs predicted values
figure;
scatter(Y, predictedY, 'filled');
hold on;
plot(Y, Y, 'r--'); % Line of perfect prediction
xlabel('Actual koi_teq');
ylabel('Predicted koi_teq');
title('Surrogate Model Verification: Actual vs Predicted');
legend('Predictions', 'Perfect Prediction');
hold off;

% Coupled Modeling and System Thinking
fprintf('\n<strong>Coupled Modeling and System Thinking</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_steff', 'koi_srad'});

% Randomly split the dataset into two parts
nRows = size(df, 1);
idx = randperm(nRows);
df1 = df(idx(1:floor(nRows/2)), :); % First half of the dataset
df2 = df(idx(floor(nRows/2) + 1:end), :); % Second half of the dataset

% Calculate the correlation in each part
correlation1 = corr(df1.koi_steff, df1.koi_srad);
correlation2 = corr(df2.koi_steff, df2.koi_srad);

% Use the average of these correlations as the expected correlation
expectedCorrelation = mean([correlation1, correlation2]);

% Display the expected correlation
disp(['Expected Correlation derived from the dataset: ', num2str(expectedCorrelation)]);

Applied Discrete and Dynamic Simulations
fprintf('\n<strong>Applied Discrete and Dynamic Simulations</strong>\n');

% Prepare the data
df = rmmissing(keplerData, 'DataVariables', 'koi_teq');

% Initial conditions
initialKoiTeq = mean(df.koi_teq); % Average temperature as starting point

% Define the hypothetical dynamic model
% For example, a simple model where temperature increases linearly over time
timeSteps = 100; % Number of time steps in the simulation
temperatureChangeRate = 0.1; % Hypothetical rate of change per time step

% Initialize array to store simulated values
simulatedKoiTeq = zeros(timeSteps, 1);
simulatedKoiTeq(1) = initialKoiTeq;

% Run the simulation
for t = 2:timeSteps
    % Update koi_teq based on the hypothetical model
    simulatedKoiTeq(t) = simulatedKoiTeq(t-1) + temperatureChangeRate;
end

% Plot the simulated koi_teq over time
figure;
plot(1:timeSteps, simulatedKoiTeq);
xlabel('Time Step');
ylabel('Simulated koi_teq');
title('Dynamic Simulation of koi_teq over Time');

% Check stability: Verify if the change is consistent with the defined rate
expectedFinalTeq = initialKoiTeq + temperatureChangeRate * (timeSteps - 1);
if abs(simulatedKoiTeq(end) - expectedFinalTeq) < 1e-3 % Tolerance for numerical stability
    disp('Simulation is stable and follows the expected pattern.');
else
    disp('Simulation shows instability or deviates from the expected pattern.');
end

% Policy and Sensitivity Analysis
fprintf('\n<strong>Policy and Sensitivity Analysis</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_impact', 'koi_teq', 'koi_period', 'koi_duration'});

% Fit a surrogate model (e.g., linear regression)
lm = fitlm(df, 'koi_teq ~ koi_impact + koi_period + koi_duration');
% Display the surrogate model
disp(lm);
% Define the range for sensitivity analysis
impactRange = linspace(min(df.koi_impact), max(df.koi_impact), 100);

% Initialize array to store predicted values
predictedTemps = zeros(length(impactRange), 1);

% Predict koi_teq for each value in impactRange
for i = 1:length(impactRange)
    % Create a table with the current value of koi_impact and mean values for other predictors
    tempTable = table(impactRange(i), mean(df.koi_period), mean(df.koi_duration), ...
                      'VariableNames', {'koi_impact', 'koi_period', 'koi_duration'});
    
    % Predict koi_teq using the surrogate model
    predictedTemps(i) = predict(lm, tempTable);
end

% Plot the sensitivity analysis results
figure;
plot(impactRange, predictedTemps);
xlabel('koi_impact');
ylabel('Predicted koi_teq');
title('Sensitivity Analysis of koi_impact on Predicted koi_teq');


% Monte Carlo Simulations
fprintf('\n<strong>Monte Carlo Simulations</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_impact', 'koi_period', 'koi_teq'});

% Fit a surrogate model (e.g., linear regression)
lm = fitlm(df, 'koi_teq ~ koi_impact + koi_period');

% Display the surrogate model
disp(lm);
% Define the number of simulations
numSimulations = 1000;

% Initialize array to store simulation outputs
simulatedOutputs = zeros(numSimulations, 1);

% Define the ranges for input parameters
impactMin = min(df.koi_impact);
impactMax = max(df.koi_impact);
periodMin = min(df.koi_period);
periodMax = max(df.koi_period);

% Run the Monte Carlo simulations
for i = 1:numSimulations
    % Randomly select values within the ranges
    randomImpact = impactMin + (impactMax - impactMin) * rand();
    randomPeriod = periodMin + (periodMax - periodMin) * rand();
    
    % Create a temporary table for prediction
    tempTable = table(randomImpact, randomPeriod, 'VariableNames', {'koi_impact', 'koi_period'});
    
    % Predict koi_teq using the surrogate model with randomized inputs
    simulatedOutputs(i) = predict(lm, tempTable);
end

% Plot the distribution of the simulated outputs
figure;
histogram(simulatedOutputs);
xlabel('Simulated koi_teq');
ylabel('Frequency');
title('Distribution of Simulated koi_teq from Monte Carlo Simulations');

Validation of Kepler’s dataset

% Validation Kepler's dataset
% Load the Kepler dataset
filePath = 'cumulative.csv'; 
keplerData = readtable(filePath);

% Introduction to Modeling for Experiments
fprintf('\n<strong>Introduction to Modeling for Experiments</strong>\n');
df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_teq'});

% Split data into training and test sets
cv = cvpartition(size(df, 1), 'HoldOut', 0.3);
idx = cv.test;
% Separate training and test data
trainData = df(~idx,:);
testData = df(idx,:);

% Train a linear regression model
lm = fitlm(trainData, 'koi_teq ~ koi_period');

% Display the model
disp(lm);

% Predict koi_teq on the test data
predictedTEQ = predict(lm, testData);

% Calculate the Mean Squared Error (MSE)
mse = mean((testData.koi_teq - predictedTEQ).^2);

% Display the MSE
disp(['Mean Squared Error: ', num2str(mse)]);

% Design of Validation Experiments
fprintf('\n<strong>Design of Validation Experiments</strong>\n');

% Assuming 'keplerData' is the dataset with the relevant fields
df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Split data into training and test sets
cv = cvpartition(size(df, 1), 'HoldOut', 0.3);
idx = cv.test;
% Separate training and test data
trainData = df(~idx,:);
testData = df(idx,:);

% Train a linear regression model
lm = fitlm(trainData, 'koi_teq ~ koi_period + koi_impact');

% Display the model
disp(lm);

% Predict koi_teq on the test data
predictedTEQ = predict(lm, testData);

% Calculate R-squared
SST = sum((testData.koi_teq - mean(testData.koi_teq)).^2);
SSR = sum((predictedTEQ - mean(testData.koi_teq)).^2);
R_squared = SSR / SST;

% Display the R-squared value
disp(['R-squared: ', num2str(R_squared)]);

% Calculate Mean Absolute Error (MAE)
MAE = mean(abs(predictedTEQ - testData.koi_teq));

% Display the MAE value
fprintf('Mean Absolute Error (MAE): %f\n', MAE);

% Calculate residuals
residuals = testData.koi_teq - predictedTEQ;

% Plot residuals
figure;
scatter(predictedTEQ, residuals);
xlabel('Predicted koi_teq');
ylabel('Residuals');
title('Residual Plot');
hline = refline([0 0]);
hline.Color = 'r';
grid on;


% Probability Modeling
fprintf('\n<strong>Probability Modeling</strong>\n');


df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Train a linear regression model
lm = fitlm(df, 'koi_teq ~ koi_period + koi_impact');

% Display the model
disp(lm);

% Calculate residuals
residuals = lm.Residuals.Raw;

% Kolmogorov-Smirnov test for normality
[h, pValue] = kstest((residuals - mean(residuals)) / std(residuals));

% Display Kolmogorov-Smirnov test results
disp(['Kolmogorov-Smirnov test result (h): ', num2str(h)]);
disp(['p-value: ', num2str(pValue)]);

% Histogram of residuals
figure;
histogram(residuals, 'Normalization', 'pdf');
xlabel('Residuals');
ylabel('Probability Density');
title('Histogram of Residuals');

% Normal distribution fit
x_values = linspace(min(residuals), max(residuals), 100);
mu = mean(residuals);
sigma = std(residuals);
normal_fit = (1 / (sigma * sqrt(2 * pi))) * exp(-0.5 * ((x_values - mu) / sigma).^2);
hold on;
plot(x_values, normal_fit, 'LineWidth', 2, 'Color', 'red');
legend('Residuals', 'Normal Fit');

% Q-Q plot of residuals
figure;
qqplot(residuals);
title('Q-Q Plot of Residuals');

% Surrogate Modeling
fprintf('\n<strong>Surrogate Modeling</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Split data into training and test sets
cv = cvpartition(size(df, 1), 'HoldOut', 0.3);
idx = cv.test;
% Separate training and test data
trainData = df(~idx,:);
testData = df(idx,:);

% Train a polynomial surrogate model
% Assuming a second-order polynomial model for demonstration
mdl = fitlm(trainData, 'koi_teq ~ koi_period^2 + koi_impact^2');

% Display the model
disp(mdl);

% Predict koi_teq on the test data using the surrogate model
predictedTEQ = predict(mdl, testData);

% Calculate Mean Squared Error (MSE) for validation
mse = mean((testData.koi_teq - predictedTEQ).^2);

% Display the MSE
disp(['Mean Squared Error (MSE): ', num2str(mse)]);

% Visualization of the model fit (Optional)
% For a 3D visualization, we can plot the observed vs. predicted values
figure;
scatter3(testData.koi_period, testData.koi_impact, testData.koi_teq, 'filled');
hold on;
scatter3(testData.koi_period, testData.koi_impact, predictedTEQ, 'filled');
xlabel('koi_period');
ylabel('koi_impact');
zlabel('koi_teq');
legend('Observed koi_teq', 'Predicted koi_teq');
title('Surrogate Model Validation');

% Multidisciplinary Modeling and Optimization
fprintf('\n<strong>Multidisciplinary Modeling and Optimization</strong>\n');


df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_duration', 'koi_teq'});

% Split data into training and test sets
cv = cvpartition(size(df, 1), 'HoldOut', 0.3);
idx = cv.test;
% Separate training and test data
trainData = df(~idx,:);
testData = df(idx,:);

% Train a linear regression model
lm = fitlm(trainData, 'koi_teq ~ koi_period + koi_impact + koi_duration');

% Display the model
disp(lm);

% Train a decision tree model
tree = fitrtree(trainData, 'koi_teq ~ koi_period + koi_impact + koi_duration');

% Predict koi_teq using both models
predictedTEQ_LM = predict(lm, testData);
predictedTEQ_Tree = predict(tree, testData);

% Combine predictions
combinedPredictions = mean([predictedTEQ_LM, predictedTEQ_Tree], 2);

% Calculate Mean Squared Error (MSE) for validation
mse = mean((testData.koi_teq - combinedPredictions).^2);

% Display the MSE
disp(['Combined Model Mean Squared Error (MSE): ', num2str(mse)]);

Coupled Modeling and System Thinking 
fprintf('\n<strong>Coupled Modeling and System Thinking</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_slogg', 'koi_srad'});

% Split data into training and test sets
cv = cvpartition(size(df, 1), 'HoldOut', 0.3);
idx = cv.test;
% Separate training and test data
trainData = df(~idx,:);
testData = df(idx,:);

% Train a linear regression model
lm = fitlm(trainData, 'koi_srad ~ koi_slogg');

% Display the model
disp(lm);

% Predict koi_srad on the test data
predictedSRAD = predict(lm, testData);

% Calculate Mean Squared Error (MSE) for validation
mse = mean((testData.koi_srad - predictedSRAD).^2);

% Display the MSE
disp(['Mean Squared Error (MSE): ', num2str(mse)]);

% Validation against astrophysical relationships
% Here, we can compare the model coefficients with known astrophysical data
% For example, compare the slope and intercept of the model with theoretical expectations
disp(['Model Coefficients: ', num2str(table2array(lm.Coefficients(:, 'Estimate'))')]);


% Applied Discrete and Dynamic Simulations
fprintf('\n<strong>Applied Discrete and Dynamic Simulations</strong>\n');

% Assuming 'koi_time0bk' as the time variable and 'koi_period' as the time-dependent variable
df = rmmissing(keplerData, 'DataVariables', {'koi_time0bk', 'koi_period'});

% For demonstration, let's assume a simple linear model
% In a real scenario, you would use a more complex time-series model
lm = fitlm(df, 'koi_period ~ koi_time0bk');

% Display the model
disp(lm);

% Predict koi_period
predictedPeriod = predict(lm, df);

% Visualization of the model fit over time
figure;
plot(df.koi_time0bk, df.koi_period, 'o');
hold on;
plot(df.koi_time0bk, predictedPeriod, '-r');
xlabel('Time (BKJD)');
ylabel('Orbital Period (days)');
title('Time-Series Model of Orbital Period');
legend('Observed', 'Predicted');

% Validation against theoretical predictions or known patterns
% Here, compare the model predictions with known astrophysical behaviors
% For example, check if the predicted periods follow Kepler's laws
% (Note: This is a simplified example. Real validation would require more detailed analysis)
disp(['Model Coefficients: ', num2str(table2array(lm.Coefficients(:, 'Estimate'))')]);

% Policy and Sensitivity Analysis
fprintf('\n<strong>Policy and Sensitivity Analysis</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Train a linear regression model
lm = fitlm(df, 'koi_teq ~ koi_period + koi_impact');

% Display the model
disp(lm);

% Sensitivity analysis on 'koi_period'
% Vary 'koi_period' within a reasonable range and observe the changes in predictions
periodRange = linspace(min(df.koi_period), max(df.koi_period), 100);
sensitivityResults = zeros(size(periodRange));
for i = 1:length(periodRange)
    testData = table(periodRange(i), mean(df.koi_impact), 'VariableNames', {'koi_period', 'koi_impact'});
    sensitivityResults(i) = predict(lm, testData);
end

% Plot the sensitivity analysis results
figure;
plot(periodRange, sensitivityResults);
xlabel('Orbital Period (days)');
ylabel('Predicted Equilibrium Temperature (koi_teq)');
title('Sensitivity Analysis on koi_period');


% Monte Carlo Simulations
fprintf('\n<strong>Monte Carlo Simulations</strong>\n');

df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Train a linear regression model
lm = fitlm(df, 'koi_teq ~ koi_period + koi_impact');

% Display the model
disp(lm);

% Monte Carlo simulation parameters
numSimulations = 1000;
simulatedPeriods = rand(numSimulations, 1) * (max(df.koi_period) - min(df.koi_period)) + min(df.koi_period);
simulatedImpacts = rand(numSimulations, 1) * (max(df.koi_impact) - min(df.koi_impact)) + min(df.koi_impact);

% Run Monte Carlo simulations
simulatedTEQ = zeros(numSimulations, 1);
for i = 1:numSimulations
    simulatedTEQ(i) = predict(lm, table(simulatedPeriods(i), simulatedImpacts(i), ...
                                        'VariableNames', {'koi_period', 'koi_impact'}));
end

% Analyze the results
figure;
histogram(simulatedTEQ);
xlabel('Predicted Equilibrium Temperature (koi_teq)');
ylabel('Frequency');
title('Monte Carlo Simulation Results for koi_teq');

Simulations

% Simulation Kepler's dfset
filePath = 'cumulative.csv';
keplerData = readtable(filePath);

% Kline-McClintock Uncertainty Propagation
fprintf('\n<strong>Kline-McClintock Uncertainty Propagation</strong>\n');

% Remove rows with missing values in relevant columns
df = rmmissing(keplerData, 'DataVariables', {'koi_period', 'koi_impact', 'koi_teq'});

% Fit a linear regression model
lm = fitlm(df, 'koi_teq ~ koi_period + koi_impact');

% Display the model
disp(lm);
% Coefficients from the linear model
beta = lm.Coefficients.Estimate';

% Define the model function using estimated coefficients
model = @(p, i) beta(1) + beta(2) * p + beta(3) * i;

% Uncertainty in inputs
uncertainty_period = std(df.koi_period);
uncertainty_impact = std(df.koi_impact);

% Sample df for derivative calculation
delta = 1e-6;
sample_df = df(1:100,:); % Sample subset for calculation

% Partial derivatives calculation
partial_period = (model(sample_df.koi_period + delta, sample_df.koi_impact) - model(sample_df.koi_period, sample_df.koi_impact)) / delta;
partial_impact = (model(sample_df.koi_period, sample_df.koi_impact + delta) - model(sample_df.koi_period, sample_df.koi_impact)) / delta;

% Uncertainty propagation
output_uncertainty = sqrt((partial_period.^2 * uncertainty_period.^2) + (partial_impact.^2 * uncertainty_impact.^2));

% Histogram of output uncertainty
histogram(output_uncertainty);
xlabel('Output Uncertainty');
ylabel('Frequency');
title('Histogram of Output Uncertainty');


% Monte Carlo Uncertainty Propagation 
fprintf('\n<strong>Monte Carlo Uncertainty Propagation</strong>\n');
num_simulations = 10000;
simulated_teq = zeros(num_simulations, 1);

for i = 1:num_simulations
    % Randomly sample a single row from the dataset
    idx = randi(height(df));
    period_sample = df.koi_period(idx);
    impact_sample = df.koi_impact(idx);

    % Add normal random noise to the samples
    period_random = period_sample + randn * uncertainty_period;
    impact_random = impact_sample + randn * uncertainty_impact;

    % Predict koi_teq for the randomized inputs
    simulated_teq(i) = model(period_random, impact_random);
end

% CDF plot of simulated koi_teq
cdfplot(simulated_teq);
xlabel('koi_teq');
ylabel('CDF');
xlim([500 1500])
title('CDF of Simulated koi_teq');

% Plot histogram of simulated koi_teq
histogram(simulated_teq);
xlabel('Simulated koi_teq');
ylabel('Frequency');
xlim([500 1500])
title('Histogram of Simulated koi_teq');

% Considering Uncertainty in Simulation 
fprintf('\n<strong>Considering Uncertainty in Simulation</strong>\n');
% Define function to simulate with uncertainty
simulate_with_uncertainty = @(p, i) model(p + randn * uncertainty_period, i + randn * uncertainty_impact);

% Apply to data
uncertain_output = arrayfun(@(p, i) simulate_with_uncertainty(p, i), df.koi_period, df.koi_impact);

% Histogram of uncertain outputs
histogram(uncertain_output);
xlabel('koi_teq with Uncertainty');
ylabel('Frequency');
xlim([500 1500])
title('Histogram of koi_teq with Uncertainty');


Confidence Interval
fprintf('\n<strong>Confidence Interval</strong>\n');
% Confidence interval calculation
ci_lower = quantile(simulated_teq, 0.025);
ci_upper = quantile(simulated_teq, 0.975);

fprintf('95%% Confidence Interval: [%f, %f]\n', ci_lower, ci_upper);


% Applying Model Bias Error 
fprintf('\n<strong>Applying Model Bias Error</strong>\n');

% Calculate model bias (hypothetical approach)
% In practice, calculate this from the difference between model predictions and actual observations
model_bias = mean(df.koi_teq) - mean(simulated_teq);

% Correct the simulated output for model bias
corrected_output = simulated_teq + model_bias;

% Plot CDF of corrected output
cdfplot(corrected_output);
xlabel('Corrected koi_teq');
ylabel('CDF');
xlim([500 1500])
title('CDF of Corrected koi_teq');


% Quantifying Sensitivity to Key Inputs 
fprintf('\n<strong>Quantifying Sensitivity to Key Inputs</strong>\n');
% Sensitivity analysis
sensitivity_period = @(p) model(p, mean(df.koi_impact));
sensitivity_impact = @(i) model(mean(df.koi_period), i);

period_range = linspace(min(df.koi_period), max(df.koi_period), 100);
impact_range = linspace(min(df.koi_impact), max(df.koi_impact), 100);

sensitivity_results_period = arrayfun(sensitivity_period, period_range);
sensitivity_results_impact = arrayfun(sensitivity_impact, impact_range);

% Plotting sensitivity
plot(period_range, sensitivity_results_period);
xlabel('koi_period');
ylabel('Predicted koi_teq');
title('Sensitivity Analysis on koi_period');

plot(impact_range, sensitivity_results_impact);
xlabel('koi_impact');
ylabel('Predicted koi_teq');
title('Sensitivity Analysis on koi_impact');


% Cumulative Distribution Function Analysis
fprintf('\n<strong>Cumulative Distribution Function Analysis</strong>\n');
% CDF plot for sensitivity analysis results
cdfplot(sensitivity_results_period);
xlabel('koi_period');
ylabel('CDF');
title('CDF of Sensitivity Analysis Results on koi_period');

cdfplot(sensitivity_results_impact);
xlabel('koi_impact');
ylabel('CDF');
title('CDF of Sensitivity Analysis Results on koi_impact');

Final Thoughts

This project’s comprehensive verification and validation processes have provided insights into the predictive modeling of exoplanetary characteristics using the Kepler dataset. The analytical techniques applied, from Kline-McClintock Uncertainty Propagation to Monte Carlo simulations, have described the complexity between the factual knowledge of orbital periods and equilibrium temperatures and the conceptual understanding of stellar-planet interactions. While the simulations have demonstrated significant underlying astrophysical principles, they have also underscored predictive models’ inherent uncertainties and sensitivities. The project’s architecture, blending statistics with astrophysical theory, shows a simulation effort guided by the importance of a multifaceted approach to astrophysical research but has also opened new pathways for inquiry, emphasizing the dynamic nature of scientific understanding in the face of uncertainty.

References

1.3.5.16. Kolmogorov-Smirnov Goodness-of-Fit Test (no date). Available at: https://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm (Accessed: 14 December 2023).

[2204.04231] Exploring and Validating Exoplanet Atmospheric Retrievals with Solar System Analog Observations (no date). Available at: https://arxiv.org/abs/2204.04231 (Accessed: 14 December 2023).

Akeson, R.L. et al. (2013) ‘The NASA Exoplanet Archive: Data and Tools for Exoplanet Research’, Publications of the Astronomical Society of the Pacific, 125(930), pp. 989–999. Available at: https://doi.org/10.1086/672273.

Ali, P. and Younas, A. (2021) ‘Understanding and interpreting regression analysis’, Evidence-Based Nursing, 24(4), pp. 116–118. Available at: https://doi.org/10.1136/ebnurs-2021-103425.

Araujo, I. (2020) Using Machine Learning to Find Exoplanets with NASA’s Dataset, Medium. Available at: https://towardsdatascience.com/using-machine-learning-to-find-exoplanets-with-nasas-dataset-bb818515e3b3 (Accessed: 14 December 2023).

Armstrong, D.J., Gamper, J. and Damoulas, T. (2021) ‘Exoplanet validation with machine learning: 50 new validated Kepler planets’, Monthly Notices of the Royal Astronomical Society, 504, pp. 5327–5344. Available at: https://doi.org/10.1093/mnras/staa2498.

Arsenault, M.-O. (2020) KOLMOGOROV–SMIRNOV TEST, Medium. Available at: https://towardsdatascience.com/kolmogorov-smirnov-test-84c92fb4158d (Accessed: 12 December 2023).

ChatGPT — Data Analysis (no date) ChatGPT. Available at: https://chat.openai.com/g/g-HMNcP6w7d-data-analysis (Accessed: 12 December 2023).

Create Cell Array — MATLAB & Simulink (no date). Available at: https://www.mathworks.com/help/matlab/matlab_prog/create-a-cell-array.html (Accessed: 12 December 2023).

Cuéllar, S. et al. (2022) ‘Deep learning exoplanets detection by combining real and synthetic data’, PLoS ONE, 17(5). Available at: https://doi.org/10.1371/journal.pone.0268199.

Descriptive Statistics and Visualization — MATLAB & Simulink (no date). Available at: https://www.mathworks.com/help/stats/exploratory-data-analysis.html (Accessed: 12 December 2023).

Difference between uncertainty intervals and confidence intervals in public health (no date) ResearchGate. Available at: https://www.researchgate.net/post/Difference_between_uncertainty_intervals_and_confidence_intervals_in_public_health2 (Accessed: 14 December 2023).

Exoplanet and Candidate Statitics (no date). Available at: https://exoplanetarchive.ipac.caltech.edu/docs/counts_detail.html (Accessed: 11 December 2023).

Exoplanet Detection on Kepler Data (no date). Available at: https://kaggle.com/code/ifteshanajnin/exoplanet-detection-on-kepler-data (Accessed: 12 December 2023).

Exoplanet Modeling and Analysis Center (no date) EMAC. Available at: https://emac.gsfc.nasa.gov (Accessed: 14 December 2023).

Exoplanet validation with machine learning: 50 new validated Kepler planets | Monthly Notices of the Royal Astronomical Society | Oxford Academic (no date). Available at: https://academic.oup.com/mnras/article/504/4/5327/5894933 (Accessed: 14 December 2023).

Exploratory Analysis of Data — MATLAB & Simulink (no date). Available at: https://www.mathworks.com/help/stats/exploratory-analysis-of-data.html (Accessed: 12 December 2023).

Exploratory Data Analysis (EDA): Types, Tools, Process (no date). Available at: https://www.knowledgehut.com/blog/data-science/eda-data-science (Accessed: 12 December 2023).

Exploratory Data Analysis (EDA) Using MATLAB: A Step-by-Step Guide | by Dr. Soumen Atta, Ph.D. | Level Up Coding (no date). Available at: https://levelup.gitconnected.com/exploratory-data-analysis-eda-using-matlab-a-step-by-step-guide-a231d37e2c9a (Accessed: 12 December 2023).

Exploratory Data Analysis using Data Visualization Techniques! — Analytics Vidhya (no date). Available at: https://www.analyticsvidhya.com/blog/2021/06/exploratory-data-analysis-using-data-visualization-techniques/ (Accessed: 12 December 2023).

Frost, J. (2017) How to Interpret P-values and Coefficients in Regression Analysis, Statistics By Jim. Available at: http://statisticsbyjim.com/regression/interpret-coefficients-p-values-regression/ (Accessed: 14 December 2023).

Full article: Exploratory Data Analysis With MATLAB (no date). Available at: https://www.tandfonline.com/doi/full/10.1080/00401706.2019.1679535 (Accessed: 12 December 2023).

Histogram with a distribution fit — MATLAB histfit (no date). Available at: https://www.mathworks.com/help/stats/histfit.html (Accessed: 14 December 2023).

Højberg, A.L. and Refsgaard, J.C. (2005) ‘Model uncertainty — parameter uncertainty versus conceptual models’, Water Science and Technology: A Journal of the International Association on Water Pollution Research, 52(6), pp. 177–186.

How do I change the font size for text in my figure? (no date). Available at: https://www.mathworks.com/matlabcentral/answers/131236-how-do-i-change-the-font-size-for-text-in-my-figure (Accessed: 12 December 2023).

How to begin axis from another value and make it a 0 point in plot… (no date). Available at: https://www.mathworks.com/matlabcentral/answers/386379-how-to-begin-axis-from-another-value-and-make-it-a-0-point-in-plot-matlab (Accessed: 12 December 2023).

How to Interpret P-values and Coefficients in Regression Analysis — Statistics By Jim (no date). Available at: https://statisticsbyjim.com/regression/interpret-coefficients-p-values-regression/ (Accessed: 13 December 2023).

How to plot a histogram as a curve? (no date). Available at: https://www.mathworks.com/matlabcentral/answers/426214-how-to-plot-a-histogram-as-a-curve (Accessed: 14 December 2023).

How to Quantify ML Model Uncertainty With Tensorflow Probability (2022) Databricks. Available at: https://www.databricks.com/blog/2022/04/28/how-wrong-is-your-model.html (Accessed: 14 December 2023).

https://www.jpl.nasa.gov (no date) Kepler — Universe Missions — NASA Jet Propulsion Laboratory, NASA Jet Propulsion Laboratory (JPL). Available at: https://www.jpl.nasa.gov/missions/kepler (Accessed: 11 December 2023).

Kepler / K2 — NASA Science (no date). Available at: https://science.nasa.gov/mission/kepler/ (Accessed: 11 December 2023).

Kepler Dataset Exploratory Analysis (no date). Available at: https://kaggle.com/code/ezietsman/kepler-dataset-exploratory-analysis (Accessed: 12 December 2023).

Kepler Exoplanet Search Results (no date). Available at: https://www.kaggle.com/datasets/nasa/kepler-exoplanet-search-results (Accessed: 11 December 2023).

Kepler Planet-Detection Mission: Introduction and First Results — NASA/ADS (no date). Available at: https://ui.adsabs.harvard.edu/abs/2010Sci...327..977B/abstract (Accessed: 11 December 2023).

Kepler Transit Graph — Exoplanet Exploration: Planets Beyond our Solar System (no date). Available at: https://exoplanets.nasa.gov/resources/1022/kepler-transit-graph/ (Accessed: 11 December 2023).

Kepler vs TESS: Telescope Comparison — Exoplanet Exploration: Planets Beyond our Solar System (no date). Available at: https://exoplanets.nasa.gov/resources/2340/kepler-vs-tess-telescope-comparison/ (Accessed: 11 December 2023).

Kepler’s legacy: discoveries and more — Exoplanet Exploration: Planets Beyond our Solar System (no date). Available at: https://exoplanets.nasa.gov/keplerscience/ (Accessed: 11 December 2023).

Kepler’s Science Results — Exoplanet Exploration: Planets Beyond our Solar System (no date). Available at: https://exoplanets.nasa.gov/resources/2189/keplers-science-results/ (Accessed: 11 December 2023).

Kirchner, M. et al. (2021) ‘Uncertainty concepts for integrated modeling — Review and application for identifying uncertainties and uncertainty propagation pathways’, Environmental Modelling & Software, 135, p. 104905. Available at: https://doi.org/10.1016/j.envsoft.2020.104905.

Kline-Mcclintock Method of Experimental Uncertainty | PDF (no date). Available at: https://www.scribd.com/doc/279352734/Kline-mcclintock-Method-of-Experimental-Uncertainty (Accessed: 14 December 2023).

Kolmogorov Smirnov Test: When and Where To Use It (no date) Arize AI. Available at: https://arize.com/blog-course/kolmogorov-smirnov-test/ (Accessed: 14 December 2023).

‘Kolmogorov–Smirnov test’ (2023) Wikipedia. Available at: https://en.wikipedia.org/w/index.php?title=Kolmogorov%E2%80%93Smirnov_test&oldid=1184191576 (Accessed: 12 December 2023).

KOLMOGOROV–SMIRNOV TEST. A needed tool in your data science… | by Marc-Olivier Arsenault | Towards Data Science (no date). Available at: https://towardsdatascience.com/kolmogorov-smirnov-test-84c92fb4158d (Accessed: 12 December 2023).

Kolodziejczak, J.J. et al. (2010) ‘Flagging and correction of pattern noise in the Kepler focal plane array’, in A.D. Holland and D.A. Dorn (eds). SPIE Astronomical Telescopes + Instrumentation, San Diego, California, USA, p. 77421G. Available at: https://doi.org/10.1117/12.857637.

Lazarus, J. et al. (no date) ‘Uncertainty Quantification: An Overview’.

Lucas, L.J., Owhadi, H. and Ortiz, M. (2008) ‘Rigorous verification, validation, uncertainty quantification and certification through concentration-of-measure inequalities’, Computer Methods in Applied Mechanics and Engineering, 197(51–52), pp. 4591–4609. Available at: https://doi.org/10.1016/j.cma.2008.06.008.

Mathur, S. et al. (2017) ‘Revised Stellar Properties of Kepler Targets for the Q1–17 (DR25) Transit Detection Run’, The Astrophysical Journal Supplement Series, 229(2), p. 30. Available at: https://doi.org/10.3847/1538-4365/229/2/30.

Methods For Incorporating Model Uncertainty Into Exoplanet Atmospheric Analysis — Astrobiology (no date). Available at: https://astrobiology.com/2023/10/methods-for-incorporating-model-uncertainty-into-exoplanet-atmospheric-analysis.html (Accessed: 14 December 2023).

Morton, T.D. (2012) ‘AN EFFICIENT AUTOMATED VALIDATION PROCEDURE FOR EXOPLANET TRANSIT CANDIDATES’, The Astrophysical Journal, 761(1), p. 6. Available at: https://doi.org/10.1088/0004-637X/761/1/6.

NASA Exoplanet Archive (2019) ‘Kepler Objects of Interest Cumulative Table’. IPAC. Available at: https://doi.org/10.26133/NEA4.

NASA’s Kepler Discovers First Earth-Size Planet In The Habitable Zone (no date) Exoplanet Exploration: Planets Beyond our Solar System. Available at: https://exoplanets.nasa.gov/resources/1063/nasas-kepler-discovers-first-earth-size-planet-in-the-habitable-zone (Accessed: 11 December 2023).

NASA’s Kepler mission doubles tally of exoplanets by weeding out impostors (2016) PBS NewsHour. Available at: https://www.pbs.org/newshour/science/nasas-kepler-mission-doubles-tally-of-exoplanets-by-weeding-out-impostors (Accessed: 12 December 2023).

Otegi, J.F. et al. (2020) ‘Impact of the measured parameters of exoplanets on the inferred internal structure’, Astronomy & Astrophysics, 640, p. A135. Available at: https://doi.org/10.1051/0004-6361/202038006.

Overview of Kepler mission (no date) Exoplanet Exploration: Planets Beyond our Solar System. Available at: https://exoplanets.nasa.gov/resources/1013/overview-of-kepler-mission (Accessed: 11 December 2023).

Peng, R.D. (no date) 6 Exploratory Graphs | Exploratory Data Analysis with R. Available at: https://bookdown.org/rdpeng/exdata/ (Accessed: 12 December 2023).

Quantification of Conceptual Model Uncertainty in the Modeling of Wet Deposited Atmospheric Pollutants — Urso — 2022 — Risk Analysis — Wiley Online Library (no date). Available at: https://onlinelibrary.wiley.com/doi/full/10.1111/risa.13807 (Accessed: 11 December 2023).

Rider, W.J. and Kamm, J.R. (no date) ‘Verification Validation and Uncertainty Quantification for CGS’.

Riesco🇫🇷, F. (2021) ‘Data Mining: Nasa Kepler Exoplanets With R’, Medium, 24 July. Available at: https://fr4nc3.medium.com/data-mining-nasa-kepler-exoplanets-with-r-3731aa50ba6a (Accessed: 13 December 2023).

Rotate x-axis tick labels — MATLAB xtickangle (no date). Available at: https://www.mathworks.com/help/matlab/ref/xtickangle.html (Accessed: 12 December 2023).

Stephanie (2022) Kolmogorov-Smirnov Goodness of Fit Test, Statistics How To. Available at: https://www.statisticshowto.com/kolmogorov-smirnov-test/ (Accessed: 12 December 2023).

Surrogate Modeling and Uncertainty Quantification — Lori Graham-Brady (no date). Available at: https://www.ce.jhu.edu/lori/surrogate-modeling-uncertainty-quantification-and-machine-learning/ (Accessed: 14 December 2023).

Symmetry | Free Full-Text | Recent Advances in Surrogate Modeling Methods for Uncertainty Quantification and Propagation (no date). Available at: https://www.mdpi.com/2073-8994/14/6/1219 (Accessed: 14 December 2023).

Tayar, J. et al. (2022) ‘A Guide to Realistic Uncertainties on the Fundamental Properties of Solar-type Exoplanet Host Stars’, The Astrophysical Journal, 927(1), p. 31. Available at: https://doi.org/10.3847/1538-4357/ac4bbc.

The MEarth Project: Science (no date). Available at: https://lweb.cfa.harvard.edu/MEarth/Science.html (Accessed: 12 December 2023).

‘The Uncertainty of Detecting Planets’ (2019) NC State News, 1 August. Available at: https://news.ncsu.edu/2019/08/uncertainty-of-detecting-planets/ (Accessed: 14 December 2023).

t-Test, Chi-Square, ANOVA, Regression, Correlation… (no date). Available at: https://datatab.net/tutorial/linear-regression (Accessed: 14 December 2023).

Uncertainty quantification patterns for multiscale models | Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences (no date). Available at: https://royalsocietypublishing.org/doi/10.1098/rsta.2020.0072 (Accessed: 14 December 2023).

Understanding and interpreting regression analysis | Evidence-Based Nursing (no date). Available at: https://ebn.bmj.com/content/24/4/116 (Accessed: 14 December 2023).

Using the Interactive Exoplanet Viewer (no date). Available at: https://exoplanetarchive.ipac.caltech.edu/docs/ICEexohelp.html (Accessed: 11 December 2023).

Valizadegan, H. et al. (2023) ‘Multiplicity Boost of Transit Signal Classifiers: Validation of 69 New Exoplanets using the Multiplicity Boost of ExoMiner’, The Astronomical Journal, 166(1), p. 28. Available at: https://doi.org/10.3847/1538-3881/acd344.

Verification and Validation and Uncertainty Quantification of Code Models: Nuclear Technology: Vol 205, No 12 (no date). Available at: https://www.tandfonline.com/doi/abs/10.1080/00295450.2019.1580532 (Accessed: 14 December 2023).

Write data to text file — MATLAB fprintf (no date). Available at: https://www.mathworks.com/help/matlab/ref/fprintf.html (Accessed: 12 December 2023).

Zhang, J. and Zhuang, J. (2017) ‘Validation, Verification, and Uncertainty Quantification for Models with Intelligent Adversaries’, in R. Ghanem, D. Higdon, and H. Owhadi (eds) Handbook of Uncertainty Quantification. Cham: Springer International Publishing, pp. 1401–1419. Available at: https://doi.org/10.1007/978-3-319-12385-1_44.

Abstract
This project explores the potential walking speed of humans on Mars, a fundamental concern in the broader context of space exploration and Martian colonization. Utilizing experimental data from Earth, we employed the Froude equation to develop a predictive model for walking speeds, which was then adjusted for Martian conditions. Recognizing the inherent uncertainties in modeling, tools such as the Kline-McClintock error propagation method and Monte Carlo simulations were harnessed. Results showed a marked difference in walking speeds on Earth versus Mars, with increased variability on the latter. Confidence intervals were generated to encapsulate this variability, with Mars exhibiting a broader span, indicating more significant uncertainty in predictions. This study exemplifies the confluence of biomechanics and space science, showing light on human adaptability on other planets and paving the way for future biomechanical research in space exploration.

Introduction

Understanding how humans would walk on the Red Planet is fundamental. This question embodies the intersection of planetary science, biomechanics, physiology, and space exploration aspirations.

When considering different gravitational environments, it becomes a complex interplay of muscle function, skeletal support, and neurophysiological feedback. Earth has conditioned our bodies to move a certain way with its unique atmospheric and gravitational properties. In the low-gravity environment of the Moon, astronauts exhibited a hopping gait. With only 38% of Earth’s gravity, Mars presents an entirely different biomechanical puzzle (Cavagna et al., 1998).

However, theoretical models are just the start. Physical experiments, computational simulations, and data from various probes and rovers add layers of complexity to our understanding. Given the resurgence in interest in Mars, fueled by governmental and private endeavors, this locomotion on the Mars question is not merely academic but essential.

Moreover, this quest is about future astronauts and humanity as a species. Understanding our essential functions on another planet becomes foundational as we set our sights on becoming an interplanetary species. Each step on Mars, each astronaut’s stride, will be a testament to human perseverance, curiosity, and our insatiable need to explore. Ultimately, the pace at which we walk on Mars is emblematic of our journey as explorers, innovators, and dreamers.

Background

Mars has been an object of human curiosity for centuries. Our understanding of this celestial body took a giant leap forward during the space age as telescopes improved and space missions were initiated. The culmination of the Moon missions, especially the sight of astronauts adapting their gait to the Moon’s lesser gravity, instigated a new realm of questions about human locomotion on other celestial bodies, particularly Mars.

While rovers like Opportunity, Spirit, and Curiosity provided invaluable data about the Martian landscape, climate, and potential for life, human adaptability remains largely theoretical. Drawing from our lunar experiences, we can make informed hypotheses. The moonwalks of the Apollo missions offered a firsthand view of human movement in a low-gravity environment. The astronauts’ “bunny hops” and bounding strides were adaptations to the Moon’s 1/6th Earth gravity (Ackermann and Van Den Bogert, 2012).

Nevertheless, Mars presents its unique challenges. With a gravity that’s 38% of Earth’s, it strikes a middle ground between the Moon and our home planet. Theoretical models, informed by lunar observations and Earth’s biomechanics, postulate a gait that’s less bounding than the Moon but with longer strides than Earth. However, the reduced Martian gravity is not the only factor. The thin atmosphere, potential for dust storms, and surface terrain variability will all influence human walking (Cavagna et al., 1998).

Recent experiments and simulations, though Earth-bound, have aimed to replicate Martian conditions. Using specially designed treadmills and harness systems that modify effective weight, scientists have begun understanding potential Martian gaits. Preliminary findings suggest increased stride lengths, modified arm swings, and altered postural dynamics. However, these Earth-based replications, while invaluable, have their limitations.

Scope

At the core of our project is the ambition to integrate these fragmented pieces of knowledge into a comprehensive model using deterministic methods. Recognizing that deterministic models offer exact predictions devoid of randomness, we aim to create a model that predicts walking speeds on Mars based on various factors, especially the often-overlooked variability in individual biomechanics like leg length.

Central to our approach is the need for validation. Using the Froude Equation and data gathered from Earth-based experiments, we aim to validate our model extensively. The equation v = T(Fr,g,L) encapsulates the essence of our approach, where v is velocity, Fr is the Froude number, g is the acceleration due to gravity, and L is the leg length. Given the known gravitational acceleration on Mars (g = 3.721m/s2), our primary variable becomes the leg length, which will be sourced from experimental data (Spilker, 2018).

Uncertainty quantification is a critical part of our scope. We recognize that no model, however refined, is immune to errors. We aim to quantify these errors from the experimentally measured leg length and the inherent tool uncertainty due to prediction errors. By propagating uncertainties from both these fronts, we aim to present an expected value for VwalkingonMars and 90% certainty bounds, providing a range of potential walking speeds.

Incorporating advanced techniques like Monte Carlo simulations and the Kline-Mclintock error propagation method, our scope expands to embrace the inherent randomness and variability in experimental data. By simulating thousands of scenarios, the Monte Carlo method will allow us to understand the probable distributions of our outcomes. On the other hand, the Kline-Mclintock method will let us propagate the uncertainties more deterministically, providing a holistic view of potential outcomes.

In conclusion, by combining rigorous scientific methods, experimental data, and simulation techniques, we hope to provide actionable insights into one of the many challenges of Mars colonization.

Figure 1: This image highlights left leg initial contact (IC), foot flat (FF), midstance (MS), heel lift (HL), and toe-off (TO) as significant phases and of importance biomechanically. Foot flat simply refers to forefoot loading, so the entire foot is in ground contact. Image Credit: footbionics.com

Figure 2: Leg Length Measurement with Tape, Image Credit: musculoskeletalkey.com

Method

Understanding how we might traverse its surface becomes essential as we stand on the brink of human exploration of the Red Planet. This study delves deep into data analytics to decode the velocity at which a human might walk on Mars. Our methodology integrates mathematical modeling with empirical data to predict walking speeds while conscientiously factoring in the uncertainties intrinsic to such research.

Our approach will use the foundational Froude equation, which postulates walking speed as a function of gravitational acceleration and leg length. While this equation offers a theoretical point of view, the real-world walking data from Earth breathes life into our models. We gain insights into human gait’s natural variability and patterns by measuring multiple subjects’ leg lengths. Nevertheless, models are rarely perfect mirrors of reality; they come with inherent assumptions and uncertainties. Recognizing this, we employ techniques like Kline-McClintock error propagation and Monte Carlo simulations, ensuring our predictions are mathematically sound and resilient to unpredictable real-world data.

With these mathematical techniques, we lean on advanced data analytics, leveraging histograms, confidence intervals, and speed discrepancy analyses to validate and refine our model. Every step has several validation procedures, from collecting terrestrial walking data to the simulations of Martian walking scenarios. These procedures ascertain the precision of our model by juxtaposing it against empirical data, thereby quantifying the inherent experimental uncertainties.

Model Assumptions and Inherent Uncertainties

In modeling walking velocity on Mars, it is essential to make certain assumptions, and it is crucial to understand the inherent uncertainties associated with these assumptions. Let us consider a simple model that calculates walking velocity as a function of leg length, assuming a constant acceleration due to gravity. Our study revolves around the Froude equation:

v = T(Fr,g,L)

The equation links gait speed, gravitational acceleration, and leg length. The gravitational acceleration is fixed for a particular celestial body. However, variations in individual leg lengths, walking patterns, and other factors create disparities between the model and experimental data. Recognizing and quantifying these deviations is essential. While Mars’ gravity is a fixed value, leg lengths and walking patterns vary across individuals. Furthermore, we have assumed that walking patterns on Mars would mirror those on Earth despite the difference in atmospheric conditions and terrain.

Speed Discrepancies

The biomechanical processes underlying our gait are profoundly ingrained and tailored to Earth’s gravitational force (approximately 99.81m/s2). When we attempt to transpose this Earth-optimized walking mechanism to Mars, whose gravity is about 33.721m/s^2, discrepancies naturally arise.

Firstly, the gravitational force on Mars, being significantly weaker than on Earth, reduces the downward force exerted on the body. This decreased force affects stride length, step frequency, and overall energy exertion. With less gravitational pull, each step can cover a more significant distance, potentially leading to longer strides. However, these longer strides might not translate directly to increased speed, as the cadence or step frequency might be altered due to changes in balance and stability under Mars’s gravity.

Secondly, the human body’s proprioceptive systems, which help us sense our body’s position and motion, have been calibrated for Earth’s conditions. The feedback loops between our muscles, nervous system, and brain, optimized for Earth, might experience a lag or misalignment on Mars. This could result in a different walking pattern or speed as the body adjusts to the new gravitational conditions.

Lastly, the Froude equation, though an excellent baseline, has roots in Earth’s biomechanics. It might only capture the whole picture when applied directly to Mars, considering the physiological and biomechanical adaptations humans undergo in reduced gravity. Factors like muscle atrophy in space, reduced bone density, and potential changes in cardiovascular health could also play a role in influencing the walking speed on Mars.

Confidence Intervals and Validation Variance

When assessing experimental data, particularly in scientific research, it is essential to understand the values we observe and how confident we are about these observations. This confidence hinges on several factors, including measurement error, biological system variability, and our models’ uncertainties.

Confidence Intervals

We have recorded walking speeds on Earth. Due to individual variability in walking speeds because of factors like age, fitness, and biomechanical differences, we observe a range of speeds even for the same leg length. By calculating the CI for this data, we obtain a range that offers insights into the natural variability of walking speeds on Earth.

While we do not have actual observational data for Mars, our simulations would also yield a range of walking speeds, informed by our model and the inherent variability we have learned from Earth data. CIs help define a range within which we expect the actual walking speeds of astronauts on Mars to fall.

Validation Variance

It focuses on the variability in our predictions and highlights the discrepancies between our simulated and observed values. This would indicate how well our biomechanical model captures real-world walking speeds for Earth. A high validation variance would suggest that our model has significant room for improvement, while a low validation variance indicates that the model predictions are close to the observed data.

When extending this concept to Mars, our validation variance would inform us about the potential discrepancies our model might exhibit when applied to Martian conditions. Given that this model is grounded in Earth-based biomechanics and observations, the validation variance for Mars would ideally incorporate adjustments or modifications based on the anticipated physiological and biomechanical changes astronauts would undergo in Mars’ gravity.

This approach ensures that our assessments for Earth and Mars are grounded in statistical rigor, offering a more reliable foundation for any subsequent analyses or predictions.

Experimental Uncertainty

Understanding experimental uncertainty is fundamental to our data-driven exploration. When collecting walking data on Earth, several sources of experimental uncertainty exist. These can stem from variability in the participants’ distractions during data collection, the measurement tools’ precision, or environmental factors like the flooring material and its frictional properties. When we transition from real-world data collection on Earth to simulating walking data on Mars, the landscape of experimental uncertainty shifts significantly. Instead of dealing with the natural variability inherent in human subjects, we grapple with uncertainties in our models, equations, and the parameters we use to simulate Martian conditions. The Froude number, the modeled gravity of Mars, and the biomechanical assumptions we input into our simulations all introduce potential uncertainties. It is worth noting that while real-world data collection has ‘pure error’, simulations possess ‘model uncertainty.’ The latter is the difference between the simulated and actual outcomes in a real Martian environment.

The ‘pure error estimate’ can be obtained for our Earth-based walking data by computing the standard deviation of repeated walking measurements for the same subject. This provides a measure of the spread of the data around the mean, reflecting the inherent variability in the subject’s walking pattern. On the other hand, for Mars, the ‘model uncertainty’ can be estimated by assessing how well our model, adjusted for Martian gravity and other factors, can predict Earth-based walking patterns. Discrepancies here can shed light on potential sources of error that might translate to our Martian simulations.

Kline-McClintock Error Propagation

Error propagation describes how uncertainties in independent variables (input) propagate to the dependent variable (output) in a given function or model. The Kline-McClintock method is a widely accepted approach, especially when addressing multiple sources of uncertainty.

Earth-Based Walking Data

When analyzing walking data on Earth, the primary sources of uncertainty stem from leg length measurements and walking speed. Variability in leg length measurements can result from different measuring techniques, human errors, or even instrument precision. Since our walking model depends on leg length, any uncertainty in this measurement can lead to errors in predicted walking speed. A partial differentiation concerning leg length (L) gives us how changes in (L) affect the walking speed (v).

Mars-Based Simulated Data

The uncertainties associated with the simulated Martian environment are introduced when extrapolating this walking data to Mars. Mars’s gravity is different from Earth’s, which affects walking speed. Additionally, the biomechanical adaptations humans might undergo while walking in reduced gravity scenarios need to be fully understood and modeled.

Using the same Froude equation but with Martian gravity:

v = T(Fr,gmars, L)

Comparing Earth and Mars

It is insightful to compare the sensitivities of walking speed to leg length for both Earth and Mars. This can highlight whether our walking model is more sensitive to leg length variations on Earth versus Mars, guiding us on where to focus our error minimization efforts.

Using the Kline-McClintock error propagation approach, we obtain a clearer picture of how uncertainties in leg length measurements can influence our predictions of walking speed. This is vital for analyzing real-world data on Earth and for the simulations that predict human movement on Mars. By comparing the two, we can gain insights into the robustness of our models and the primary sources of potential discrepancies.

Monte Carlo Simulation for Uncertainty Analysis

Monte Carlo simulations evaluate a range of inputs to provide a probability distribution of potential outcomes. This probabilistic approach gives information and captures inherent variability and uncertainties present in real-world scenarios.

Transition to Mars-Based Simulated Data

When projecting this data onto Mars, gravity is the primary change. The Froude equation models walking speed as a function of leg length and gravity. By applying Monte Carlo simulations, we can consider the uncertainties in our model, leg lengths, and any other parameters we are unsure of, such as biomechanical adaptations humans might undergo in the Martian environment.

Comparative Insights

Comparing the histograms of walking speeds on Earth and Mars, we might observe shifts in the central tendencies or wider spreads in one scenario versus the other. This tells us not only about the effects of gravity but also about the inherent uncertainties when modeling unfamiliar environments like Mars.
The Monte Carlo simulation allows us to harness the power of randomness to study the variability in our data and models. Such simulations offer valuable insights into the probable walking scenarios we might encounter, uncommonly when projecting known experimental data onto an unknown environment like Mars.

Walking Velocity Data Collection on Earth

The transition from walking to running is not arbitrary. It is primarily governed by biomechanics and energy optimization. On Earth, various factors, including the length of one’s legs, determine this transition point. The longer the legs, the larger the step and, often, the higher the speed at which one transitions from walking to running. By gathering data on leg length and walking velocity, we aim to develop a biomechanical model grounded in real-world observations. This data collection becomes our benchmark.

Data Characteristics on Earth

Each entry in our Earth dataset represents a unique combination of leg length and walking velocity. Given that we have twelve entries, it implies we have collected data from 12 distinct individuals. This dataset captures the natural variability among humans. For instance, two individuals with the same leg length might still have different walking speeds due to fitness level, age, or specific biomechanical nuances.

Simulating Walking Data on Mars

When transitioning to Mars, we can only partially take and scale the Earth’s speeds directly. As mentioned earlier, Mars has approximately 38% of Earth’s gravity. This will inevitably affect the biomechanics of walking. By feeding our Earth data into the Froude equation modified for Martian gravity, we can simulate what walking speeds might look like on Mars for the same set of leg lengths.

The Value of Earth Data

With the Earth data as a reference, any simulation for Mars would be plausible. Earth data provides a necessary anchoring point, ensuring our simulations are rooted in reality. It is like understanding walking dynamics in a familiar environment like Earth before predicting it in an unfamiliar one like Mars.

In essence, the walking velocity data collected on Earth is a critical calibration tool for our Mars walking model. It gives us a validation mechanism and the essential parameters to fine-tune our simulations for another planetary environment.

Table 1: Experimental data

Results

Experimental Data Uncertainty

The formulas we utilize are rooted in biomechanics, blending the principles of human movement with the fundamental laws of physics. Central to our calculations is the Froude equation, which ties together the gait speed, gravitational acceleration, and leg length, offering an approximation of walking speed under various gravitational conditions. This equation provides the foundational model for our predictions.

However, every model needs to be revised. We contrast these predictions against real-world data collected from human subjects to understand their accuracy and relevance. The subsequent formulas, ranging from error estimations to experimental uncertainties, serve dual purposes. First, they quantify the divergence between our model’s theoretical outputs and actual human performance. Second, they measure the reliability and variability of our experimental data, ensuring we are not just capturing anomalous behavior but a representative sample of human walking patterns. In essence, these formulas offer a rigorous mathematical lens through which we assess the efficacy and precision of our biomechanical walking model.

Uncertainty in the Model of Walking Velocity

The uncertainty in the walking velocity model is the difference between the velocities predicted by our mathematical model (using the Froude equation) and the velocities we observe in real-world data. This difference indicates how accurate or off-mark our theoretical predictions are compared to real-world walking speeds.

Formula:

Result:

Model Uncertainty: 0.267815 m/s

Estimated Error in the Walking Model

The estimated error quantifies how the predicted velocities from the walking model deviate from actual observed speeds. This is determined by calculating the relative difference between the model’s predictions and the observed data and then averaging over all data points.

Formula:

Result:

Mean Relative Error in Walking Model: 0.134450

Measured Uncertainty in the Experiment

refers to the inherent variability or spread within the collected walking speed data. The standard deviation of observed speeds reflects natural differences in subjects’ walking patterns.

Formula:

Result:

Pure Error Estimate (Measured Uncertainty in the Experiment): 0.240715 m/s

Error in the Walking Model

This is the overall discrepancy between our theoretical predictions from the model and the actual experimental observations. It can be considered an average “miss” of our model over all data points.

Formula:

Result:

Total Error in Walking Model: 0.357953 m/s

Validation Against Experimental Mean

The average walking speed from the model predictions is compared to the average speed from the real-world data. It measures how close our model's "central tendency" is to that of the actual data.

Formula:

Result:

Validation Error Against Experimental Mean: 0.351439 m/s

Experimental Uncertainty

This metric provides an insight into the experimental measurements' spread out or variable. It is given by the coefficient of variation, which is the standard deviation expressed as a percentage of the mean, thus making it a relative measure of dispersion.

Formula:

Result:
Experimental Uncertainty (Coefficient of Variation): 0.094821

Confidence Interval

For the 90% confidence interval (CI) for the mean error in our experimental data, we typically employ the t-distribution since our sample size is relatively small (n < 30). The formula for the confidence interval for the mean of a sample from a normally distributed population when the sample size is small.

The 90% confidence interval for the mean error of our experimental data is plotted, and a histogram of the relative errors is plotted. The histogram lines indicate the mean error and the lower and upper bounds of the 90% confidence interval.

90% CI for Earth walking speeds: [0.091976, 0.17692] m/s

Mars walking simulation

By leveraging models like the Froude equation alongside real-world data on Earth, this simulation will show the difference between our current terrestrial understanding and the impending Martian expeditions.
Kline-McClintock Uncertainty Propagation

The Kline-McClintock method for uncertainty propagation quantifies the uncertainty of an output variable (in this case, walking speed on Mars) based on the uncertainties in the input variables like the Froude number, gravitational acceleration, and leg length.

Formula:

Result:

Monte Carlo Uncertainty Propagation

We use the Monte Carlo simulations to run the model 10,000 times with leg length sampled from their respective probability distributions. After many runs, we will have a distribution of walking speeds, and we can analyze this distribution to understand the variability in the model output due to uncertainties in the inputs.

Considering Uncertainty in Simulation

The relative uncertainty remains constant during the transition from Earth to Mars. This assumption can be made based on the information available for this experiment.

Uncertainty in Leg Length

From the given data, we can calculate the leg length’s standard deviation (or variance) as our measure of uncertainty.

Formula:

Result:
Leg variance = 0.0070

Distribution of Walking Speed on Mars

The Cumulative Distribution Function (CDF) of the potential walking speeds on Mars provides a comprehensive view of the distribution of walking speeds on Mars, helping understand the range, variability, and likelihood of different speeds.

Confidence Interval

We use percentiles from the data to calculate the 90% confidence interval (CI) for the Mars walking speeds obtained from the simulation. The 5th and 95th percentiles represent the bounds of the 90% CI. The plot displays the distribution of the simulated Mars walking speeds along with vertical dashed lines representing the 90% confidence interval. The area between these two dashed lines contains 90% of all simulated speeds.
90% CI for Mars walking speeds: [0.90549, 1.0467] m/s

Discussion

Our analysis revolved around understanding walking speeds on both Earth and Mars. This involved knowing how to interpret, process, and use the experimental data. Recalling the mathematical formulations such as the Froude equation, we tapped into the foundational theories of biomechanics. The CDF allowed us to understand the probability distribution of our walking speeds, offering a comprehensive view of our data. In our models, we quantified various aspects of uncertainty. The model’s mean relative error (0.134450) and the pure error estimate (0.240715 m/s) specifically delineated how our model corresponded to real-world observations. The CDF was pivotal as a foundation for our Monte Carlo simulations guiding our random sampling processes.
The 90% CI for walking speeds on Earth is [0.091976, 0.17692] m/s. This suggests that, based on our Earth data and at a 90% confidence level, we expect the average walking speed to fall within this range. The span of this interval is approximately 0.08494 m/s, a measure of the uncertainty inherent in our Earth-based model.

For Mars, the 90% CI for walking speeds becomes [0.90549, 1.0467] m/s. An immediate observation is that the speeds on Mars, as projected by the model, are significantly higher than those on Earth. This reflects Mars’s different conditions, such as gravity and biomechanical constraints, compared to Earth’s. However, the span of this interval is approximately 0.1412 m/s, which is broader than the Earth’s CI span.

The broader range for Mars suggests increased uncertainty in our Martian predictions compared to our Earth-based ones. It is not surprising given that our Martian model is a projection based on Earth data and involves additional assumptions about conditions on Mars. We need to learn more about walking on Mars than on Earth, which can introduce more variability into the model.

Regarding the model’s performance, we unearthed that the total error in the Earth walking model was 0.357953 m/s. Breaking this down, the validation error against the experimental mean was 0.351439 m/s. The CDF offered a visual representation of this error distribution, allowing the interpretations of where and how our model deviated from experimental results.

The potential walking speeds on Mars are created by combining the insights from our Earth data, Martian constraints, and the probability distributions offered by the CDFs. This allows us a comprehensive view of how terrestrial biomechanics might manifest in an extraterrestrial setting.

The tools and methodologies employed were crucial in offering predictions and establishing these forecasts’ reliability. Future studies could refine this approach by delving into more sophisticated probabilistic models or integrating other biomechanical data sources.

In conclusion, this simulation provides a framework for understanding and predicting human walking speeds on Mars by applying deterministic and probabilistic methods coupled with the insights offered by CDFs. To perfect these models, we need to gather more data and insights. Our predictions will only become more refined and reliable.

Matlab Code

% Given experimental data
data = [0.927 2.778
        0.965 3.148
        0.927 2.380
        0.938 2.288
        1.178 2.364
        1.066 2.450
        1.027 2.717
        0.881 2.610
        0.909 2.380
        0.909 2.427
        0.97 27.475
        1.016 2.448];


leg_length = data(:,1);
observed_speed = data(:,2);
g_earth = 9.80665; % acceleration due to gravity on Earth
Fr = 0.5; % Froude number for walking

% 1. Uncertainty in Model of Walking Velocity
modeled_speed = sqrt(Fr * g_earth * leg_length);
model_uncertainty = std(modeled_speed - observed_speed);

% 2. Estimated Error in the Walking Model
relative_error_model = abs(modeled_speed - observed_speed) ./ observed_speed;
mean_relative_error_model = mean(relative_error_model);

% 3. Measured Uncertainty in the Experiment (Pure Error Estimate)
pure_error_estimate = std(observed_speed);

% 4. Error in the Walking Model
total_model_error = mean(abs(modeled_speed - observed_speed));

% 5. Validation Against Experimental Mean
experimental_mean = mean(observed_speed);
validation_error = abs(experimental_mean - mean(modeled_speed));

% 6. Experimental Uncertainty
experimental_uncertainty = std(observed_speed) / mean(observed_speed); % Coefficient of variation

% Printing the results
fprintf('1. Model Uncertainty: %f m/s\n', model_uncertainty);
fprintf('2. Mean Relative Error in Walking Model: %f\n', mean_relative_error_model);
fprintf('3. Pure Error Estimate (Measured Uncertainty in the Experiment): %f m/s\n', pure_error_estimate);
fprintf('4. Total Error in Walking Model: %f m/s\n', total_model_error);
fprintf('5. Validation Error Against Experimental Mean: %f m/s\n', validation_error);
fprintf('6. Experimental Uncertainty (Coefficient of Variation): %f\n', experimental_uncertainty);



% 1. Uncertainty in Model of Walking Velocity:

% Calculate modeled walking speeds

% Plotting
figure;
plot(leg_length, observed_speed, 'o', leg_length, modeled_speed, '-');
legend('Observed Speed', 'Model Speed');
title('Observed vs. Modeled Walking Speed');
xlabel('Leg Length (m)');
ylabel('Walking Speed (m/s)');


%2. Estimated Error in the Walking Model:

% Histogram of relative error
figure;
histogram(relative_error_model);
title('Histogram of Relative Errors');
xlabel('Relative Error');
ylabel('Frequency');


%3. Measured Uncertainty in the Experiment:

% Histogram of speeds
figure;
histogram(observed_speed);
title('Histogram of Observed Speeds');
xlabel('Speed (m/s)');
ylabel('Frequency');

%4. Error in the Walking Model:

% Plotting error between observed and modeled speeds
figure;
plot(leg_length, observed_speed - modeled_speed, 'o-');
title('Error in Walking Model');
xlabel('Leg Length (m)');
ylabel('Error (m/s)');


%5. Validation Against Experimental Mean:

% Plotting observed vs. mean speed
figure;
plot(leg_length, observed_speed, 'o', leg_length, repmat(experimental_mean, length(leg_length), 1), '--');
legend('Observed Speed', 'Mean Speed');
title('Validation Against Experimental Mean');
xlabel('Leg Length (m)');
ylabel('Walking Speed (m/s)');

%6. Experimental Uncertainty:

% Plotting speed with error bars (standard deviation)
figure;
errorbar(leg_length, observed_speed, pure_error_estimate * ones(size(observed_speed)), 'o');
title('Experimental Uncertainty in Observed Speeds');
xlabel('Leg Length (m)');
ylabel('Walking Speed (m/s)');


% Compute the mean and standard deviation of the errors
mean_error = mean(relative_error_model);
std_error = std(relative_error_model);

% Sample size
n = length(relative_error_model);

% Compute the t critical value for 90% CI
alpha = 0.10;
t_critical = tinv(1 - alpha/2, n-1);

% Compute the margin of error
margin_error = t_critical * (std_error/sqrt(n))

% Confidence Interval
CI_lower = mean_error - margin_error
CI_upper = mean_error + margin_error

disp(['90% CI for Earth walking speeds: [', num2str(CI_lower), ', ', num2str(CI_upper), '] m/s']);

% Plot histogram with mean error and confidence intervals
histogram(error, 'Normalization', 'probability');
hold on;
xline(mean_error, '-r', 'Mean Error');
xline(CI_lower, '--g', 'Lower 90% CI');
xline(CI_upper, '--g', 'Upper 90% CI');
hold off;
title('Histogram of Relative Errors with Mean and 90% CI');
xlabel('Relative Error');
ylabel('Probability');
legend('Error Distribution', 'Mean Error', '90% CI');

% Mars Simulation

g_mars = 3.721;  % Mars gravitational acceleration
Fr = 0.25;

% 1. Kline-McClintock Uncertainty Propagation:
dv_dL = Fr * g_mars / (2 * sqrt(Fr * g_mars * mean(leg_length)))
sigma_L = std(leg_length)
sigma_v = dv_dL * sigma_L % propagated uncertainty

% 2. Monte Carlo Uncertainty Propagation:
num_simulations = 10000;
v_mars_simulated = zeros(num_simulations, 1);

for i = 1:num_simulations
    L_sample = datasample(leg_length, 1);  % sample one leg length from Earth data
    v_mars_simulated(i) = sqrt(Fr * g_mars * L_sample);
end

% Plot histogram of simulated walking speeds on Mars
histogram(v_mars_simulated);
title('Distribution of Walking Speeds on Mars');
xlabel('Walking Speed (m/s)');
ylabel('Frequency');

% Calculate the histogram of v_mars_simulated data with 'Normalization' option set to 'cdf'
[num_counts, bin_edges] = histcounts(v_mars_simulated, 'Normalization', 'cdf');
randomnums = rand(num_simulations, 1);
% The CDF values are represented by 'num_counts', and the corresponding walking speed values (on Mars) are represented by the midpoints of 'bin_edges'
bin_centers = (bin_edges(1:end-1) + bin_edges(2:end)) / 2;


% Plot the CDF
figure;
plot(bin_centers, num_counts, 'LineWidth', 2);
xlabel('Walking Speed on Mars (m/s)');
ylabel('CDF');
title('Cumulative Distribution Function of Walking Speed on Mars');
grid on;


% Calculate the 90% CI
lower_bound = prctile(v_mars_simulated, 5);
upper_bound = prctile(v_mars_simulated, 95);

% Display
disp(['90% CI for Mars walking speeds: [', num2str(lower_bound), ', ', num2str(upper_bound), '] m/s']);

% Plot histogram and the CI
figure;
histogram(v_mars_simulated, 50, 'Normalization', 'pdf'); % Change 50 if you want a different number of bins
hold on;
y_limits = ylim;
plot([lower_bound, lower_bound], y_limits, 'r--', 'LineWidth', 2);
plot([upper_bound, upper_bound], y_limits, 'r--', 'LineWidth', 2);
title('Histogram of Simulated Mars Walking Speeds with 90% CI');
xlabel('Walking Speed (m/s)');
ylabel('Probability Density');
legend('Simulated Speeds', '90% CI');
grid on;
hold off;


% Calculating leg length variance

leg_variance = var(leg_length)
% Simulating walking speed on Mars using the Froude equation
v_mars = sqrt(Fr * g_mars * leg_length);

% Histograms and Plots
figure;
histogram(leg_length);
title('Leg Length Distribution');

References

Ackermann, M. and Van Den Bogert, A.J. (2012) ‘Predictive simulation of gait at low gravity reveals skipping as the preferred locomotion strategy’, Journal of Biomechanics, 45(7), pp. 1293–1298. Available at: https://doi.org/10.1016/j.jbiomech.2012.01.029.

Average Speed or Pace Calculator (no date). Available at: http://www.datedial.com/datAverage_Speed_Calculator.asp (Accessed: 17 October 2023).

Behavioral Statistics in Action (no date). Available at: https://www.palomar.edu/users/rmorrissette/Lectures/Stats/Variability/Variability.htm (Accessed: 17 October 2023).

Can You Walk On Mars? (ANSWERED) — USVAO (2022). Available at: https://usvao.org/can-you-walk-on-mars/ (Accessed: 14 October 2023).

Cavagna, G.A., Willems, P.A. and Heglund, N.C. (1998) ‘Walking on Mars’, Nature, 393(6686), pp. 636–636. Available at: https://doi.org/10.1038/31374.

Farrington, R.B. and Wells, C.V. (no date) ‘A Thorough Approach to Measurement Uncertainty Analysis Applied to Immersed Heat Exchanger Testing’.

Froude number (Fr) | Britannica (no date). Available at: https://www.britannica.com/science/Froude-number (Accessed: 17 October 2023).

Hawkey, A. (2004) ‘Small step or giant leap? Human locomotion on Mars’, Journal of the British Interplanetary Society, 57(7–8), pp. 262–270.

Hinson-Williams, J. (no date) Libraries: Writing an Educational Research Paper: Research Paper Sections. Available at: https://libguides.bc.edu/edpaper/sections (Accessed: 16 October 2023).

How long would it take to walk to mars from earth? (no date) Answers. Available at: https://www.answers.com/astronomy/How_long_would_it_take_to_walk_to_mars_from_earth (Accessed: 14 October 2023).

How will we walk on Mars? (no date). Available at: https://www.esa.int/ESA_Multimedia/Images/2014/09/How_will_we_walk_on_Mars (Accessed: 14 October 2023).

mars.nasa.gov (no date) Moving around Mars — NASA. Available at: https://mars.nasa.gov/mer/mission/timeline/surfaceops/navigation/ (Accessed: 14 October 2023).

Mitchell, J. (2021) Predicting Disneyland Wait Times through Population Simulations, Medium. Available at: https://towardsdatascience.com/predicting-disneyland-wait-times-through-population-simulations-20f44c7582f6 (Accessed: 14 October 2023).

‘Random walk’ (2023) Wikipedia. Available at: https://en.wikipedia.org/w/index.php?title=Random_walk&oldid=1174474148 (Accessed: 14 October 2023).

Research Paper Structure (no date). Available at: https://psychology.ucsd.edu/undergraduate-program/undergraduate-resources/academic-writing-resources/writing-research-papers/research-paper-structure.html (Accessed: 16 October 2023).

Society, T.P. (2018a) ‘The First Mars Marathon: Part 1’, The Physiological Society, 29 August. Available at: https://www.physoc.org/blog/the-first-mars-marathon-part-1/ (Accessed: 17 October 2023).

Society, T.P. (2018b) ‘The First Mars Marathon: Part 2’, The Physiological Society, 5 September. Available at: https://www.physoc.org/blog/the-first-mars-marathon-part-2/ (Accessed: 17 October 2023).

Society, T.P. (2018c) ‘The First Mars Marathon: Part 3’, The Physiological Society, 7 September. Available at: https://www.physoc.org/blog/the-first-mars-marathon-part-3/ (Accessed: 17 October 2023).

Spilker, T. (2018) ‘Answer to “What would the human gait look like on Mars?”’, Space Exploration Stack Exchange. Available at: https://space.stackexchange.com/a/32867 (Accessed: 14 October 2023).

‘True and Apparent Leg Length Measurement | Bone and Spine’ (2017), 16 August. Available at: https://boneandspine.com/true-and-apparent-leg-length/ (Accessed: 22 October 2023).

Uncertainty Calculator (no date). Available at: https://uncertaintycalculator.com/ (Accessed: 22 October 2023).

walking speed estimation: Topics by Science.gov (no date). Available at: https://www.science.gov/topicpages/w/walking+speed+estimation (Accessed: 14 October 2023).

What Is a Confidence Interval and How Do You Calculate It? (no date) Investopedia. Available at: https://www.investopedia.com/terms/c/confidenceinterval.asp (Accessed: 22 October 2023).

When data centers are run in-house, it is impossible to build redundancy without paying for extra infrastructure. With cloud infrastructure, however, multiple options are available at a lower price point. Moreover, high-risk activities such as moving core networks pose a lesser threat because the need for interfering with physical infrastructure decreases almost to zero.

Introduction

With new technologies and cheaper cloud services, companies can decide to move all their services and data storage from an on-premises data center to a cloud computing service provider. Having their services in an on-premise data center has its risks. For example, a physical data center has power cords and cables that can present trip hazards. Also, the data center has environmental hazards as risk factors include heat and cold, hurricanes, and earthquakes that can affect your ability to provide your services. In addition, those risks associated with the location and data centers also have similar risks to cloud services, such as DDoS attacks and phishing, to name a few. This document aims to perform risk analysis and risk mitigation tasks to reduce losses and migrate smoothly from an on-premises data center to a cloud computing infrastructure.

During a data center to cloud migration, there are a lot of things that can go wrong. These cloud migration problems can be time-consuming, less efficient, and have downtime for the company. Moreover, we need to recreate the infrastructure to support our current project and services, and we need to back-up databases and transfer them to the new cloud platform. Lastly, we must decommission the on-premise servers and data from the old data center.

FAIR Analysis

Migrating from an on-premises data center to a cloud infrastructure requires several activities that can bring risks during their execution. Some of these migration activities are:

• Application migration involves service migration to a new cloud infrastructure, web applications, backend API, compliance and risk applications, etc.
• Database migration involves migrating from one database to another vendor database or consolidating different databases into one or an open-source big data platform.
• Hardware migration includes decommissioning any hardware that is no longer in use.

Loss Event Frequency

The number of times a data center will be migrated to a cloud infrastructure will effectively cause losses in our business. Controls on having a detailed plan, a time framework, data backup of the systems, and a rollback plan in case the migration is compromised and we can reduce costs in the loss event.

Threat Event Frequency,

The number of times a data center will be migrated to a cloud infrastructure will happen in a year. Not all migrations will produce losses in our business and our services.

Contact Frequency

The number of times the threat agents come close to putting our assets in danger during the migration.

Probability of Action

The probability that the threat agent puts at risk the success of the migration process.

Vulnerability

When creating the new cloud infrastructure is used with poor configuration, shared credentials between services, and databases with ports exposed to the internet, among other vulnerabilities. Moreover, the databases that need to migrate from a data center to the cloud, the data in transit, and the initial location and final destination. Any service exposed to the internet creates vulnerabilities in our migration process. Controls need to be applied, such as cloud configuration audits, databases fully encrypted, and special credentials for the process.

Threat Capability

Threat agents can exploit any step of the migration process. The database can be exposed to malicious actors or unauthorized users during the migration preparation or during the execution. The services aren’t doesn’t have the same behavior as the original services, thus causing widespread disruption.

Difficulty

To difficult any vulnerability during the migration execution, we should have a detailed plan for the migration, the team in charge of any implementation, the date for testing new infrastructure before the migration, and a plan to protect the data at any moment. We should have up-to-date system versions and patches in the new infrastructure. Also, create credentials access for each database and system, i.e., don’t share Credentials between systems and infrastructure layers.

Loss Magnitude

The loss created by the migration process, such as downtime, loss of the capability to provide services, and private data exposure, among others.

Primary Loss

The significant migration process, such as loss in productivity, cannot provide services to our customers and clients. Another loss factor is the Response and putting back the costs of the services.

Secondary Risk

The secondary loss in the migration process is associated with fines for breaking service-level agreements (SLA), reputation damage, delay to delivery and contract constraints, and privacy liability for data breaches.

Secondary Loss Event

Fines for breaking SLA can occur only when the migration process affects our services for a prolonged time. We can have compromised data that cannot be recovered. We could mitigate this cost or losses if all data associated with our business is always encrypted and backed up regularly and have a redundancy infrastructure.

Secondary Loss Magnitude

This can be calculated based on the cost of notification of customers and stakeholders, contractual fines for SLA infringement, and public relations costs, among other costs.

Cost-Effective Cloud Migration

Deciding to migrate from a data center to a cloud infrastructure often faces significant challenges when it comes to cost. In many cases, the upfront investment required to make a move is unviable, making it difficult to realize the long-term benefits of the cloud. Anyway, there are several ways that we can use to ensure a cost-effective migration process. The most important is identifying which company assets can be moved to the cloud. By carefully planning our migration and taking advantage of available resources and technology, we can ensure a cost-effective transition to the cloud.

Risk Simulation Analysis

Model 1: LEF (PERT), PL (PERT), and SL (constant)

We created a Monte Carlo risk analysis simulation using the FAIR model implemented in python. We use the PERT distribution to model our risk analysis. This distribution is from a family of continuous probability distributions defined by the minimum, most likely, and maximum values that a variable can take to represent the uncertainty of the value of some quantities, such as Primary Loss, Secondary Loss, Loss Magnitude, and Lost Event Frequency, to name a few, because they are subjective estimates from our analysis. In our first model, the parameters for the Loss Event Frequency (LEF) occurs won’t happen in a year and is most likely to happen only once a year and at most. Also, it could occur one time per year. Primary losses (PL) are typically where we put losses incurred directly by our company. In our simulation after the analysis, we estimated the minimum primary loss could be 1M, most likely 10M, and in a complete disaster, 200M; Secondary Losses (SL) are typically where we put losses that are caused by actions that secondary stakeholders might take. In this case, we define if secondary losses will be a constant loss of 1M.

Simulation Results

Model 2: LEF (Normal) and LM (PERT)

In our second model, the parameters for the Loss Event Frequency (LEF) are calculated as a normal distribution with a mean of 0.3 and a standard deviation of 0.1. For the Loss Magnitude (LM), we used the PERT distribution. And in our simulation after the analysis, we estimated the minimum LM could be 1M, most likely 10M, and in a complete disaster, 200M;

Simulation results.

In our first model, the results show the minimum loss is $457,288. The maximum loss is $183,189,653. The mean Loss is $33,892,168. The second model shows that the minimum losses are $1,210,228. The maximum loss is $194,875,225. The mean loss is $45,874,807. While we know these results are not predictions, they are probabilities that, even with a 1% of possibility, could still happen. Moreover, the assumptions made to calculate these risk scenarios are based on our estimates of possible losses. These simulations can help us identify which assets need to be secured to reduce any possible loss.

Risk Control Assessment

Migrating from an on-premises data center to a cloud infrastructure is a very complex process that needs planning, a roadmap, an evaluation of the risk of each stage, a rollout strategy, a rollback strategy, a security plan, and a risk mitigation plan for each of the stage. During the migration process, our team has to overcome several risks to execute the migration process.

Infrastructure migration

One critical risk is when we build the new infrastructure in our new cloud services. We don’t only need to recreate all the existing services, but also, we need to implement the security associated with our new virtual home. We need to enable risk mitigation control. To start, we need to establish baseline configurations and regularly conduct configuration auditing to check for any security flaws in the network creation, network connectivity, enabling public and private subnetworks, security groups, and data storage and databases layer, should not have access to the internet, among other restrictions. Another risk mitigation for the migration process is limiting access to different layers, subnetworks, and other places for employees. For example, we need to implement a zero-trust model and reduce any risk that bad actors can access our network and put our services at risk. Moreover, all users should have strong passwords, no share accounts, and use MFA to login into our infrastructure. Lastly, we must implement a plan to test the newly created services. For example, we need to strengthen our services’ configuration, such as a full proxy for HTTP, firewall, Virtual private network, and subnet private for database layers, server performance anomaly detection, SSL renegotiation validation, rate limiting, and strict TCP forwarding. Performing load testing and security testing on our new premises will help mitigate downtime when moving to the new environment.

Data Migration

Data migration is one of the critical factors for the datacenter migration. This transformation involves technology taking data from our data center legacy system and putting it into our new cloud infrastructure. It is essential to extract the data and determine what can be migrated, what’s missing, what needs to be cleaned, and what transformation is required to fit the new environment. To ensure that the data migration is successfully executed, we must establish robust reconciliation processes to help mitigate any technical risk. Creating an orchestrated strategy to translate the databases from our legacy systems will help identify problems when they occur. We should build migration data routines that we need to test to identify any pitfalls in the process and keep our data integrity. Another risk mitigation that needs to be in place during the migration process is to create a critical and rigorous process when the data is in transit and at rest. This means end-to-end encryption, secure storage, and limited access to remove any data breach. Lastly, everybody involved in the data migration process should have defined credentials with specific access levels. We should keep credentials individually by users and service users and verify and validate the security in each step of the data migration private.

Legacy Datacenter

Migrating from the on-premises data center to cloud infrastructure will leave a trace of unused resources that need to be decommissioned, recycled, or destroyed depending on the material, the use, and the level of privacy associated with the items. To reduce any risk with the leftover of our migration, we need to create a data center decommissioning plan. First, we need to develop an assets inventory such as cabling, boxing, packages, servers, docking seatings, keyboards, racks, and power cords, to name a few. Any physical asset in the data center that doesn’t require security decommission can be recycled, reused, sold, or given to charities. Another risk mitigation we need to implement is a comprehensive backup plan for the database, data servers, and data storage decommission. When we are sure the data is secured in its new location, we propose physical destruction to protect sensitive data or work with a security company to help us permanently erase our legacy data center servers and wipe & sanitize physical infrastructure.

References

(5) Managing the Risk of Data Centre Migration | LinkedIn (no date). Available at: https://www.linkedin.com/pulse/managing-risk-data-centre-migration-jake-holloway/ (Accessed: 15 December 2022).

5 Steps for a Successful On-Premise to Cloud Migration (no date). Available at: https://www.teradata.com/Trends/Cloud/On-Premises-to-Cloud-Migration (Accessed: 15 December 2022).

Admin, S. (2021) Data Center Strategy & Its Effect on Customer Experience, TRG Datacenters. Available at: https://www.trgdatacenters.com/data-center-strategy/ (Accessed: 15 December 2022).

Agee, B. (no date) FAIR Terminology 101 — Risk, Threat Event Frequency, Vulnerability. Available at: https://www.fairinstitute.org/blog/fair-terminology-101-risk-threat-event-frequency-and-vulnerability (Accessed: 15 December 2022).

Akinrolabu, O. (no date) ‘Cyber Supply Chain Risks in Cloud Computing — The Effect of Transparency on the Risk Assessment of SaaS Applications’.

AWS Partner Webcast — Data Center Migration to the AWS Cloud: A Customer’s Experience (2014). Available at: https://www.youtube.com/watch?v=jE1-8vjMGrw (Accessed: 15 December 2022).

Bartlett, R. (no date) Data Center Relocation Risk Assessment. Available at: https://blog.triparagon.com/data-center-relocation-risk-assessment (Accessed: 15 December 2022).

Beka, M.-M. and Gritzalis, S. (no date) ‘Cyber Risk Management for data-driven enterprises’.

Best practices for data center risk assessment | TechTarget (no date) Data Center. Available at: https://www.techtarget.com/searchdatacenter/tip/Best-practices-for-data-center-risk-assessment (Accessed: 14 December 2022).

Case Study: How FAIR Risk Quantification Enables Information Security Decisions at Swisscom (no date) ISACA. Available at: https://www.isaca.org/resources/isaca-journal/issues/2020/volume-5/how-fair-risk-quantification-enables (Accessed: 15 December 2022).

Copeland, J.B. (2022) ‘Build or Buy an Application to Run FAIR Cyber Risk Quantification?’, Security Boulevard, 16 June. Available at: https://securityboulevard.com/2022/06/build-or-buy-an-application-to-run-fair-cyber-risk-quantification/ (Accessed: 15 December 2022).

Copeland, J.B. (no date a) 7 Basic Tools for FAIR Cyber Risk Analysis. Available at: https://www.fairinstitute.org/blog/7-basic-tools-for-fair-cyber-risk-analysis (Accessed: 14 December 2022).

Copeland, J.B. (no date b) A New Approach to Data for Faster FAIR Quantitative Risk Analysis. Available at: https://www.fairinstitute.org/blog/a-new-approach-to-data-for-faster-fair-quantitative-risk-analysis (Accessed: 15 December 2022).

Cyber Event Risk Quantification: 4 Common TEF Mistakes (Web Apps) (no date). Available at: https://www.risklens.com/resource-center/blog/common-mistakes-calculating-event-frequency-web-apps (Accessed: 15 December 2022).

Data Center Migration: 7 Most Common Cloud Migration Problems (2017) Exit Technologies. Available at: https://www.exittechnologies.com/blog/cloud-computing/data-center-migration-7-common-cloud-migration-errors/ (Accessed: 15 December 2022).

‘Data Center Migration and Risk Mitigation Assessment’ (no date).

Data center migration to cloud (no date) Google Cloud. Available at: https://cloud.google.com/solutions/migration-center (Accessed: 15 December 2022).

Data center migration to cloud | Google Cloud (no date). Available at: https://cloud.google.com/solutions/migration-center (Accessed: 9 December 2022).

Data Center Threats and Vulnerabilities (no date) Check Point Software. Available at: https://www.checkpoint.com/cyber-hub/cyber-security/what-is-data-center/data-center-threats-and-vulnerabilities/ (Accessed: 15 December 2022).

Data Center vs Cloud — What’s the Difference? (no date) Check Point Software. Available at: https://www.checkpoint.com/cyber-hub/cyber-security/what-is-data-center/data-center-vs-cloud/ (Accessed: 15 December 2022).

Data Migration Challenges and solution for successful implementation | LinkedIn (no date). Available at: https://www.linkedin.com/pulse/20140918151302-65816706-data-migration-challenges-and-solution-for-successful-implementation/ (Accessed: 15 December 2022).

EdPrice-MSFT (no date) Extend on-premises data solutions to the cloud — Azure Architecture Center. Available at: https://learn.microsoft.com/en-us/azure/architecture/data-guide/scenarios/hybrid-on-premises-and-cloud (Accessed: 15 December 2022).

FAIR for Quantitative Risk (2018) Kindly Ops. Available at: https://www.kindlyops.com/knowledge-base/fair-example/ (Accessed: 14 December 2022).

How to ace on premise to cloud migration in 2021 (no date). Available at: https://www.keboola.com/blog/how-to-ace-on-premise-to-cloud-migration-in-2021 (Accessed: 15 December 2022).

How to Estimate and Manage Your Data Migration Costs | Arcion (no date). Available at: https://www.arcion.io//learn/data-migration-costs (Accessed: 14 December 2022).

Hussein, A.A. (2020) ‘Data Migration Need, Strategy, Challenges, Methodology, Categories, Risks, Uses with Cloud Computing, and Improvements in Its Using with Cloud Using Suggested Proposed Model (DMig 1)’, Journal of Information Security, 12(1), pp. 79–103. Available at: https://doi.org/10.4236/jis.2021.121004.

Implementing A Data Center Relocation Method — NCWS (no date) National Computer Warehouse Services, LLC. Available at: https://nationalcws.com/ (Accessed: 15 December 2022).

Institute, F. (no date) Cost-Effective Risk Management | Resources. Available at: https://www.fairinstitute.org/learn-fair (Accessed: 14 December 2022).

joshuanatan (2020) ‘Quantitative Risk Assessment Using FAIR’, The Startup, 28 November. Available at: https://medium.com/swlh/quantitative-risk-assessment-using-fair-313ca0f4b1ef (Accessed: 15 December 2022).

Lamba, A.J., Ankush (no date) How to Mitigate the Risks and Challenges in Data Migration, Passle. Available at: https://angle.ankura.com//post/102hjcr/how-to-mitigate-the-risks-and-challenges-in-data-migration (Accessed: 15 December 2022).

Lever, M. (2020) Data Center Migration — Best Practices, Silverback Data Center Solutions. Available at: https://teamsilverback.com/data-center-migration-best-practices/ (Accessed: 15 December 2022).

Merritt, R. (no date) Anatomy of a FAIR Risk Analysis: Confidential Data in Email. Available at: https://www.fairinstitute.org/blog/anatomy-of-a-fair-risk-analysis-confidential-data-in-email (Accessed: 15 December 2022).

Michael, K. (2012) ‘Security Risk Management: Building an Information Security Risk Management Program from the Ground Up’, Computers & Security, 31(2), pp. 249–250. Available at: https://doi.org/10.1016/j.cose.2011.12.011.

Misra, S. (no date) Prioritize Cloud Security Controls with FAIR. Available at: https://www.fairinstitute.org/blog/prioritize-cloud-security-controls-with-fair (Accessed: 15 December 2022).

Molden, M. (2022) System migration: 7 risks to evaluate, The Hyland Blog. Available at: https://blog.hyland.com/digital-transformation/system-migration-risks-to-evaluate/ (Accessed: 15 December 2022).

Monte Carlo Simulation 101 in 5 Minutes (no date). Available at: https://www.risklens.com/videos/monte-carlo-simulation-101-in-5-minutes-video (Accessed: 14 December 2022).

Naunheim, T. (2022) ‘pyfair’. Available at: https://github.com/theonaunheim/pyfair/blob/f7008c1ab3df24200dd194886e466f077066c23c/docs/getting_started.rst (Accessed: 15 December 2022).

‘PERT distribution’ (2022) Wikipedia. Available at: https://en.wikipedia.org/w/index.php?title=PERT_distribution&oldid=1100235293 (Accessed: 15 December 2022).

Prijic, M. (2022) ‘How to Avoid Data Center Migration Challenges’, IT Convergence, 5 June. Available at: https://www.itconvergence.com/blog/avoiding-challenges-during-data-center-migration/ (Accessed: 15 December 2022).

Rastogi, I., Chandra, A. and Singh, A. (2013) ‘Cloud Security Risk Assessment using FAIR’, 4(1).

RiskLens Unveils a New Triage Function for Rapid Risk Quantification (no date). Available at: https://www.risklens.com/resource-center/blog/risklens-unveils-a-new-triage-function-for-rapid-risk-quantification (Accessed: 15 December 2022).

Rogier, B. (2016) ‘How to mitigate performance risk in a data center migration?’, Accedian, 18 May. Available at: https://accedian.com/blog/data-center-migration-mitigating-risk/ (Accessed: 15 December 2022).

RPubs — Example FAIR calculations (no date). Available at: https://rpubs.com/emurphy77/fair-example (Accessed: 14 December 2022).

Security, R.S.I. (2020) ‘The Basics to Completing a FAIR Assessment’, RSI Security, 4 June. Available at: https://blog.rsisecurity.com/the-basics-to-completing-a-fair-assessment/ (Accessed: 15 December 2022).

Security, R.S.I. (2022) ‘Best Practices for Auditing the Cloud’, RSI Security, 20 September. Available at: https://blog.rsisecurity.com/best-practices-for-auditing-the-cloud/ (Accessed: 15 December 2022).

Snoweb (no date) FAIRTM️ risk methodology: quantifying and managing cyber risk. Available at: https://www.c-risk.com/en/blog/fair-analysis/ (Accessed: 15 December 2022).

Strategies for Cost Reduction and Risk Management (no date). Available at: https://www.thomasnet.com/insights/imt/2009/05/19/strategies-for-cost-reduction-and-risk-management-in-the-purchasing-department/ (Accessed: 14 December 2022).

Sudhan, M. (2022) ‘Why is it Better to Migrate On-Premise Data to Cloud’, Cloud Data Analytics Company, 28 November. Available at: https://www.anblicks.com/blog/why-is-it-better-to-migrate-on-premise-data-to-cloud/ (Accessed: 15 December 2022).

Sweeney, A. (no date a) 6 Data Center Migration Challenges and How to Overcome Them. Available at: https://www.readyworks.com/blog/6-data-center-migration-challenges-and-how-to-overcome-them (Accessed: 15 December 2022).

Sweeney, A. (no date b) Data Center Migration: Best Practices to Reduce Risk. Available at: https://www.readyworks.com/blog/data-center-migration-best-practices-to-reduce-risk (Accessed: 15 December 2022).

Three Steps to Evaluate Security Risks of Cloud Migration (no date). Available at: https://www.risklens.com/resource-center/blog/three-steps-to-evaluate-security-risks-of-cloud-migration (Accessed: 15 December 2022).

Why RiskLens? | RiskLens (no date). Available at: https://www.risklens.com/company/why-risklens (Accessed: 15 December 2022).

Wiswell, S. (no date a) ‘Data Center Migration’, p. 67.

Summary

Ransomware Attacks can have different origins. For instance, cybercriminals target the business cloud infrastructure, or a user receives phishing with ransomware and infects her workstation and the company’s intranet network. Therefore, we have different ways to model a FAIR assessment, avoid and prevent attacks, and reduce risk and damage.

Ransomware Evolution

Ransomware is a computer program created by a criminal to control a whole computer system or the data located in a personal computer, company network, or cloud services. After a system is attacked by Ransomware, the cybercriminal demand ransom money. Moreover, if the demands are not met, the data cannot be accessed because the malware software encrypts it. As we can see in figure 1, since the inception of Ransomware for that 30 years ago, Ransomware has evolved from a simple floppy disk-distributed malware infection that blocked or interfered with normal computer functions to dangerous data encryption that demands ransom payments to recover the data or unblock the complete computer functionality. The ransomware attacks’ sophistication started in the early 2010s when more robust encrypted algorithms were created. By 2016, Ransomware as a service (RaaS) was created, where cybercriminals with no programming skills could rent services for their organized crime. This same year, new Ransomware was written on javascript opening the door to target Linux, mac, and windows machines.

In 2017, the most potent Ransomware attack was perpetrated. The WannaCry ransomware attack has victims in more than 150 countries. The wannacry, also a ransomworm, was spread via an eternal blue vulnerability. New ransomware attacks evolved through the trend of improving existing Ransomware with new variants that have complex algorithms rather than creating new strains. We still need to figure out what is next for Ransomware. However, cybercriminals and malicious actors will continue to create more effective and efficient malware to encrypt files and request ransom money.

Risks in Ransomware attacks

Loss Event Frequency

The number of times Ransomware will effectively cause losses in our business. Controls on having a backup of the systems, the product assembly line, and data compromised can reduce costs in the loss event.

Threat Event Frequency,

The number of times Ransomware will happen in a year. Not all Ransomware attacks will produce losses in our business and our services.

Contact Frequency

The number of times the threat agents trigger scripts that scan to check systems for vulnerabilities services access to configuration.

Probability of Action

The probability that the threat agent initiates a Ransomware attack after the initial contact with our cloud services.

Vulnerability

Cloud infrastructure is used with poor configuration, shared credentials between services, and databases with ports exposed to the internet, supply chain computers using an obsolete version of the software, or not with the latest security patch, among other vulnerabilities.

Any service exposing its configuration access to the internet creates vulnerabilities to our infrastructure.

Threat Capability

Threat agents can exploit database access with default credentials, network sniffing, remote site access, or brute force. Then, after gaining access, replace our resources with encrypted data across the entire company network, and services, thus causing widespread disruption.

Difficulty

To difficult any ransomware attack, we should have up-to-date the system version and patches. Also, create credentials access for each database and system, i.e., don’t share Credentials between systems and infrastructure layers. Moreover, hardware assembly and blueprint should be protected and have a backup to avoid discontinuing the production chain. Databases should have access only by a specific subnetwork, and they should always be in a private subnetwork that cannot access directly from the internet, among other vulnerabilities control.

Loss Magnitude

The loss created by ransomware attacks variates from if we pay the ransom, loss of the capability to provide services, and private data exposure, among others.

Primary Loss

The significant losses of ransomware attacks are loss in productivity, such as employees being unable to use their workstations where we cannot provide services to our customers and clients, and assembly hardware disruption. Another loss factor is the Response and putting back the costs of the services.

Secondary Risk

The secondary loss in ransomware risk is associated with fines for breaking service-level agreements (SLA), reputation damage, delay to delivery and contract constraints, and privacy liability for data breaches.

Secondary Loss Event

Fines for breaking SLA can occur only when the ransomware attack affects our services for a prolonged time. The compromised data cannot be recovered, and the assembly line cannot be returned to work. We could mitigate this frequency of cost or losses if all data associated with our business is always encrypted and backed up regularly.

Secondary Loss Magnitude

This can be calculated based on the cost of notification of customers and stakeholders, contractual fines for SLA infringement, and public relations costs, among other costs.

Prevent, Minimize and Detect Ransomware

When we think of ransomware attacks, the worth case scenario is to pay the ransom, and we will get our data or systems back. Sadly, this is not true. Cybersecurity companies and the FBI oppose paying ransom to this criminal. Not only is it playing their game, but also, it does not warrant that you will get your data back. For that reason, the best way to protect our companies and us from ransomware attacks is to prevent attacks, educate employees, users, and customers about security risks, comply with access endpoints, and constantly backup up the company’s digital information and users’ data. First, backup sensitive data. This data should be appropriately stored and be sure that a cyber attack cannot target it. Also, we need to be sure that the backup information is not infected and can roll during a data recovery plan. Another efficient and easy way to protect the company and our personal information against ransomware is to keep the company, employees, and users’ workstation systems up to date. This means any application, laptop, workstation, or IoT system should run with the latest version and all the security patches available for their specific model and software version. Lastly, employees and users should be trained in cybersecurity threads and how to avoid being a victim of a ransom attack. These required users to avoid and flag malicious emails, malware software detection running on a workstation, and malware attack monitoring in the company network. All these steps are necessary to minimize the risk of a ransomware attack and help to recover from a successful attack.

Your Part in ransomware attacks

Paying ransomware attacks will allow new and future malicious agents to try more sophisticated and complex ransomware attacks. The possibility of getting money from afraid companies that can lose more if they cannot provide their services is very profitable. With that scenario, my first approach is to try to remove the ransomware from systems and roll back a recovery plan. As security IT and risk management experts, we should know the tradeoff of losing productivity, inability to sell products, and how long the company can be without providing its services to full capacity. Therefore, if the backup and recovery process cannot recover the company’s full capacity and ability to provide our services. We can recall the Colonial Pipeline ransomware attack. This company mainly carries gasoline and jet fuel to the Southeastern United States, and they suffered a ransomware attack that impacted computerized equipment managing critical infrastructure. Because fuel is one of the essential materials for almost every business, it was critical to recover access to Colonial Pipeline and be able to continue transporting/transferring fuel. The company paid 4.4 Billion in ransom to reduce increased pressure to resolve the issue promptly. Ultimately, this decision should be considered critical before paying any ransom. However, paying a ransom should be the last resource, and it should be coordinated with the criminal prosecutor to bring to justice all the malicious agents that find this type of crime an opportunity.

References

7 Steps to Help Prevent & Limit the Impact of Ransomware (2020) CIS. Available at: https://www.cisecurity.org/blog/7-steps-to-help-prevent-limit-the-impact-of-ransomware/ (Accessed: 31 October 2022).

Colonial Pipeline hack explained: Everything you need to know (no date). Available at: https://www.techtarget.com/whatis/feature/Colonial-Pipeline-hack-explained-Everything-you-need-to-know (Accessed: 31 October 2022).

Colonial Pipeline ransomware attack — Wikipedia (no date). Available at: https://en.wikipedia.org/wiki/Colonial_Pipeline_ransomware_attack (Accessed: 31 October 2022).

Copeland, J.B. (no date) Ransomware Risk: Setting Up a FAIR Analysis. Available at: https://www.fairinstitute.org/blog/ransomware-risk-setting-up-a-fair-analysis (Accessed: 31 October 2022).

Countering ransomware: Lessons from aircraft hijacking — Atlantic Council (no date). Available at: https://www.atlanticcouncil.org/in-depth-research-reports/issue-brief/countering-ransomware-lessons-from-aircraft-hijacking/ (Accessed: 31 October 2022).

eschroeder (2021) ‘Countering ransomware: Lessons from aircraft hijacking’, Atlantic Council, 26 August. Available at: https://www.atlanticcouncil.org/in-depth-research-reports/issue-brief/countering-ransomware-lessons-from-aircraft-hijacking/ (Accessed: 31 October 2022).

Fig. 2. Evolution of Major Ransomware Families from 1989 to 2020. (no date) ResearchGate. Available at: https://www.researchgate.net/figure/Evolution-of-Major-Ransomware-Families-from-1989-to-2020_fig2_349310347 (Accessed: 30 October 2022).

Follin, L. (2022) ‘Ransomware Attacks: How They Happen, the Threats and Risks.’, Pentest People, 3 May. Available at: https://www.pentestpeople.com/ransomware-attacks-how-they-happen-the-threats-and-risks/ (Accessed: 30 October 2022).

Irwin, L. (2021) The 5 biggest ransomware pay-outs of all time, IT Governance UK Blog. Available at: https://www.itgovernance.co.uk/blog/the-5-biggest-ransomware-pay-outs-of-all-time (Accessed: 31 October 2022).

Ransomware 101 Part 1: A Growing Threat to Financial Services Firms (no date). Available at: https://www.acaglobal.com/insights/ransomware-101-part-1-growing-threat-financial-services-firms (Accessed: 31 October 2022).

Ransomware attack risks (no date). Available at: https://advisory.kpmg.us/articles/2021/ransomware-attack-risks.html (Accessed: 31 October 2022).

The history and evolution of ransomware (no date) SearchSecurity. Available at: https://www.techtarget.com/searchsecurity/feature/The-history-and-evolution-of-ransomware (Accessed: 30 October 2022).

Vendor ransomware attack impacts 660 healthcare organizations (no date). Available at: https://www.fiercehealthcare.com/health-tech/more-600-providers-impacted-ransomware-attack-payment-vendor (Accessed: 31 October 2022).

Vulnerabilities Allow Hijacking of Most Ransomware to Prevent File Encryption | SecurityWeek.Com (no date). Available at: https://www.securityweek.com/vulnerabilities-allow-hijacking-most-ransomware-prevent-file-encryption (Accessed: 31 October 2022).

A Brief Story About Tracking on the Internet

The tracking of the consumer on the web started in the 90s when the internet became popular in many households. At this point, the newborn eCommerce websites needed to distinguish between consumers and identify what they were looking for in their sites. That is how the HTTP cookies were created. This simple technology allowed the websites to assign unique cookies for each customer on their site to store their session and know who they were and what they were browsing. There was nothing fancy about HTTP cookies; they were issues by each site and could not be shared between sites. But that was not the end of the story; tech companies identified the millions of possibilities to create a personalized experience based on user habits on different sites (The Evolution of the Internet, Lab, n.d.). As a software developer in the early 2000s, I saw and coded this kind of ad. For example, a third-party company wanted their ads on our site. Therefore, we passed basic information from our customers to their ads, such as gender, age, and location, to name a few tags; in return, the ad system showed an ad targetted for that specific customer group. Similarly, we have the same ad services to create targeted ads to bring traffic to our site. As simple as it started, this evolved from a permanent tracking of everything action of the customer on the internet to a more deeply invasion for the name of a personalized internet experience.

The Loss of Our Innocence and Our Privacy

Everything was fine when personalizing advertising started, but the technology grew. The digital footprint of consumers created on the internet is quadruplicated with the help of new services, and how users utilized multiple devices on the internet. Free services, online games, and social media have become popular and used by almost everybody, from teens to grandparents, and all these services need to monetize their products. To handle this immense volume of data created, the technology became faster, more accurate, and more sophisticated to collect, transfer, and store this information. All these investments needed to bring revenues, so companies started to develop more complex algorithms, not only for target ads but also for political purposes, among other invasive targeting, approaches. At this point, collecting the browsing activities of the consumers was not sufficient. Therefore, the tech companies moved to more intruding techniques by using audio devices and recording conversations to create targeting features based on these invasive approaches. If that was not enough, the companies that saw value in collecting their customer data to provide a personalized experience decided to sell this information and generate extra income from the resources they collected. With the lack of privacy laws or loopholes on them, the customers were left helpless in this wild west of the personal data collection.

The Champions of Data Privacy

In this rampage of data collection and privacy loopholes, consumers’ data can be collected and passed from multiple companies without the user’s consent. Companies that originally collect data from their consumers, they were able to sell their consumers’ data to third-party companies that don’t even have the same terms and conditions that the consumers may agree to in the beginning. Although these practices are questionable, consumers are not always protected. Sadly, it depends on where the customers live and where the data is collected. This means customers’ data are under different laws, and each company has different privacy terms and conditions. Consequently, ethics of data manipulation fall under the good intentions of the people who work doing data collection and data mining.
Let’s start with the privacy laws that protect consumers in the US. For instance, it does not exist a standardized national law that forces companies to notify their clients of any data breach. Also, companies can sell their customers’ private data to third parties, and these companies can resell the data again without consent or even inform you. Additionally, the US has several laws for data privacy, and these laws are for different types of data, such as the Health Insurance Portability and Accountability Act (HIPAA) for patient and doctor confidentiality and; the Fair Credit Reporting Act (FCRA), among others. That is why the US has several loopholes that allow companies to use their customers’ data without any scruple. In contrast, the European Union has robust privacy laws called General Data Protection Regulation (GDPR). The GDPR forces companies to have formal permission to share any information of their clients, and the individuals have the right to request to access and delete their data (The State of Consumer Data Privacy Laws in the US, 2021).

With all the ambiguities in the data protection laws, the data engineers, data scientists, and software engineers have the moral and ethical duties to advocate for better laws to protect the customers and their privacy and ensure that the data that we gathered is appropriately handled. To illustrate, any company that collects and manages users’ information should explicitly request consent to store and use personal information. All data collected should be protected when it is in transit and at rest. This means end-to-end encryption, secure storage, and limited access to remove any data breach. Suppose the data is sold to third parties. In that case, it should be already clean of any personal information that allows identifying the user identity. We need to stand up and be the champions for data privacy and promote laws that protect users’ privacy. This will allow the future of data science to be accurate, safe, and trustworthy.

Conclusion

There can be no doubt that data mining for business has changed our consumer experience for the better. Getting a tailored experience for our everyday needs simplifies our daily routine and safe us money and time. But this entails opening our privacy and being subject to deeper surveillance that must be controlled, limited, and audited. For that reason, consumers, companies, and governments have to work together to protect the data and use it for the welfare of everybody on the internet.

References

Accuracy measures for a forecast model — Accuracy. (n.d.). Retrieved May 9, 2022, from https://pkg.robjhyndman.com/forecast/reference/accuracy.html

Become A Data Privacy Week Champion. (n.d.). Stay Safe Online. Retrieved May 13, 2022, from https://staysafeonline.org/data-privacy-week/become-dpw-champion/

Carpenter, A. (2020, August 10). The Ethics of Data Collection. Medium. https://towardsdatascience.com/the-ethics-of-data-collection-9573dc0ae240

Cate, F. H. (n.d.). Government Data Mining: The Need for a Legal Framework. Harvard Civil Rights, 43, 57.

Cross-site tracking — Read our definition on the tea house. (n.d.). The Tea House by Fifty-Five. Retrieved May 11, 2022, from https://teahouse.fifty-five.com/en/glossary/cross-site-tracking/

Data protection. (n.d.). [Text]. European Commission — European Commission. Retrieved May 13, 2022, from https://ec.europa.eu/info/law/law-topic/data-protection_en

Data Protection and Privacy: 12 Ways to Protect User Data. (n.d.). Cloudian. Retrieved May 12, 2022, from https://cloudian.com/guides/data-protection/data-protection-and-privacy-7-ways-to-protect-user-data/

Data protection in the EU. (n.d.). [Text]. European Commission — European Commission. Retrieved May 13, 2022, from https://ec.europa.eu/info/law/law-topic/data-protection/data-protection-eu_en

DataPlanet — What Are My Ethical and Legal Responsibilities in Using Data and Statistics. (n.d.). Retrieved May 11, 2022, from https://dataplanet.sagepub.com/data-basics/what-are-my-responsibilities-in-using-data-and-statistics

DuckDuckGo. (2022). In Wikipedia. https://en.wikipedia.org/w/index.php?title=DuckDuckGo&oldid=1086836111

Duhigg, C. (2012, February 16). How Companies Learn Your Secrets. The New York Times. https://www.nytimes.com/2012/02/19/magazine/shopping-habits.html

Google won’t read Gmail emails anymore for advertisement — GHacks Tech News. (2017, June 23). GHacks Technology News. https://www.ghacks.net/2017/06/23/google-wont-read-gmail-emails-anymore-for-advertisement/

Google-run’awayer. (2021, November 8). 7 Best Private Search Engines that won’t track you like Google does. Comparitech. https://www.comparitech.com/blog/vpn-privacy/best-private-search-engine/

Haselton, T. (2017, December 6). How to find out what Google knows about you and limit the data it collects. CNBC. https://www.cnbc.com/2017/11/20/what-does-google-know-about-me.html

Helft, M., & Vega, T. (2010, August 30). Retargeting Ads Follow Surfers to Other Sites. The New York Times. https://www.nytimes.com/2010/08/30/technology/30adstalk.html

How to Notify Your Customers of Your Privacy Practices, and What Not to Do. (2017, September 9). Woopra. https://www.woopra.com/blog/how-to-notify-your-customers-of-your-privacy-practices-and-what-not-to-do

Kantor, J. (2014, August 13). Working Anything but 9 to 5. The New York Times. https://www.nytimes.com/interactive/2014/08/13/us/starbucks-workers-scheduling-hours.html, https://www.nytimes.com/interactive/2014/08/13/us/starbucks-workers-scheduling-hours.html

Legal and Ethical Issues in Obtaining and Sharing Information. (n.d.). Morris Manning & Martin, LLP. Retrieved May 13, 2022, from https://www.mmmlaw.com/media/legal-and-ethical-issues-in-obtaining-and-sharing-information/

Nield, D. (n.d.). All the Ways Google Tracks You — And How to Stop It. Wired. Retrieved May 11, 2022, from https://www.wired.com/story/google-tracks-you-privacy/

Optimizing Your Ads for Google’s Mobile Search Pages. (2019, August 8). Metric Theory. https://metrictheory.com/blog/optimizing-your-ads-for-googles-mobile-search-pages/

Schofield, J. (2018, April 19). What’s the best email service that doesn’t scan emails for ad-targeting? The Guardian. https://www.theguardian.com/technology/askjack/2018/apr/19/whats-the-best-email-service-that-doesnt-scan-emails-for-ad-targeting

The Evolution of the Internet, Identity, Privacy and Tracking — How Cookies and Tracking Exploded, and Why We Need New Standards for Consumer Privacy — IAB Tech Lab. (n.d.). Retrieved May 11, 2022, from https://iabtechlab.com/blog/evolution-of-internet-identity-privacy-tracking/

The State of Consumer Data Privacy Laws in the US (And Why It Matters). (2021, September 6). Wirecutter: Reviews for the Real World. https://www.nytimes.com/wirecutter/blog/state-of-privacy-laws-in-us/

Title Capitalization Tool — Capitalize My Title — Title Case Tool. (n.d.). Capitalize My Title. Retrieved May 11, 2022, from https://capitalizemytitle.com/

What is Consumer Privacy and Which Laws Protect It? (n.d.). SearchDataManagement. Retrieved May 13, 2022, from https://www.techtarget.com/searchdatamanagement/definition/consumer-privacy

What is Data in Transit and Data at Rest. (n.d.). Retrieved May 12, 2022, from https://www.quest-technology-group.com/academy/what-is-data-in-transit-vs-data-at-rest

What is End-to-End Encryption. (n.d.). Retrieved May 12, 2022, from https://www.quest-technology-group.com/academy/what-is-end-to-end-encryption

What Is SEO / Search Engine Optimization? (n.d.). Search Engine Land. Retrieved May 12, 2022, from https://searchengineland.com/guide/what-is-seo

What Search Engines Don’t Track You? (2021, October 20). Brave Browser. https://brave.com/learn/no-tracking-search-engine/

(N.d.-a). Retrieved May 11, 2022, from https://www.metarouter.io/blog-posts/the-ethics-of-collecting-consumer-data

(N.d.-b). Retrieved May 13, 2022, from https://www.metarouter.io/blog-posts/the-ethics-of-collecting-consumer-data

The most distant galaxy observed and confirmed by the NASA Ultra Deep Field project is GN-z11 with a redshift of 11.1 or 13.39 billion years away and almost 200 million years closer to the Big Bang. Images from Hubble and Spitzer show that GN-z11 is 25 times smaller than the Milky Way and approximately. However, GN-z11 is very small, it is forming new stars with a ratio of 20 times more than our galaxy (NASAWeb-I)

Figure 1: Image from the farthest galaxy GN-z11 superimposed on an image from the GOODS-North survey.

Credits: NASA, ESA, P. Oesch and B. Robertson (University of California, Santa Cruz), and A. Feild (STScI) (NASAWeb-I)

Figure 2: infographic image from farthest galaxies known so far. Hubble spectroscopically confirms the farthest galaxy to date.

Credits: NASA, ESA, P. Oesch and B. Robertson (University of California, Santa Cruz), and A. Feild (STScI) (NASAWeb-I)

Cosmic microwave background (CMB)

Cosmic microwave background (CMB) image was observed by the space telescope Planck (Figure 1). This image is a snapshot from the early universe when the universe was 380,000 years old. In the image, the universe displays small temperature variations from regions where the densities change (ESAWeb-I).

The Planck space telescope is observing the universe at wavelengths between 0.3 mm to 11.1 mm which are the frequencies 27Ghz to 1 THz (far-infrared, microwave, and high-frequency radio domains). The principal objective of this telescope is to study CMB which are remains of radiation left from the Big Bang. The CMB is leftover radiation from when the universe was created about fourteen billion years ago. The presence of CMB was proposed by George Gamow, Ralph Alpher, and Robert Herman in the 40s when they were studying the nucleosynthesis of elements such as hydrogen, helium, and lithium. The CMB was detected for the first time in 1964 by Arno Penzias and Robert Wilson. They won the Nobel Prize in Physics in 1978 for their discovery. (ESAWeb-I)

Before the Planck space telescope, NASA sent the first space mission to record the CMB in 1989. This mission discovered that CMB has a spectrum of a blackbody with temperatures of 2.73 kelvin. For those discoveries, John Mather and George Smoot got the Nobel Prize in Physics in 2006. In 2001, NASA sent a second space mission which is known as Wilkinson Microwave Anisotropy Probe (WMAP), this mission studied the small variations in temperature and they discovered that these very small variations in temperature in the CMB were left by the matter and photons unpair 380,000 years after the Big Bang. WMAP helped to identify the components of the universe and helped to build the modern model of cosmology. (ESAWeb-I)

Figure 3: Planck CMB, Released 21/03/2013, Credit: ESA and the Planck Collaboration (ESAWeb-I)

Abell 370

On May 4, 2017, Nasa released the image of Abell 370, a galaxy cluster composed of more than a hundred galaxies all bound together by gravity. This cluster is located six billion light-years away from us in the Cetus Constellation. Abell 370 is so massive that it causes the light of background galaxies to become anamorphic and amplified (Figure 1). This phenomenon produced by the massive gravitational influence of the cluster creates a magnified effect that helps astronomers with better visualization of the galaxies behind and farther away than the cluster(NASAWeb-I).

This image of Abell 370 was taken by the Hubble Space telescope after more than 600 hours of observation time and orbits of the Earth, respectively. This observation is part of the Frontier Fields program which is a collaborative effort by NASA and ESA. The main objective of this program is to perform the deepest observations of the universe in order to help astronomers understand how stars and galaxies evolved after the Big Bang when space was opaque and repleted with Hydrogen (NASAWeb-I).

Moreover, in these deep field observations, Hubble also computed six parallel fields close to Abell 370 (Figure 2). These observations are helping astronomers to understand the distributions of normal and dark matter in those clusters. Lastly, the most important part of this release demonstrated that astronomers now have a full dataset of these clusters which is the most complete dataset of the early universe ever captured (NASAWeb-I).

Figure 4: Abell 370, a galaxy cluster Hubble Space Telescope

Credit: NASA, ESA, and J. Lotz and the HFF Team (STScI) NASAWeb-I

Figure 5: Abell 370 parallel field, imaged taken by a Hubble Deep Field

Credit: NASA, ESA/Hubble, HST Frontier Fields NASAWeb-I

References

NasaWeb-I Morrow, A. 2016, NASA, http://www.nasa.gov/feature/goddard/2016/hubble-team-breaks-cosmic-distance-record (Accessed May 7, 2017)

ESAWeb-I European Space Agency, http://www.esa.int/Our_Activities/Space_Science/Planck/Planck_and_the_cosmic_microwave_background (Accessed May 7, 2017)

Boylan-Kolchin, M., Weisz, D. R., Bullock, J. S., & Cooper, M. C. 2016, Mon Not R Astron Soc Lett, 462, L51
Dunlop, J. S., McLure, R. J., Biggs, A. D., et al. 2017, Mon Not R Astron Soc, 466, 861
ESOWeb-I www.spacetelescope.org, http://www.spacetelescope.org/news/heic1711/ (accessed May 7 2017)
NASAWeb-I Hille, K. 2017, NASA, http://www.nasa.gov/feature/goddard/2017/a-lot-of-galaxies-need-guarding-in-this-hubble-view (accessed May 7 2017)
SpaceWeb-I Weitering, H., 5, S. W.-P. | M., & ET, 2017 Space.com, http://www.space.com/36726-hubble-galaxies-need-guarding.html (accessed May 7 2017)
Umehata, H., Tamura, Y., Kohno, K., et al. 2017, Astrophys J, 835, 98