Develop a web-based service that allows users to upload product images and apply realistic lighting effects using pre-trained IC-Light models. The service would offer a range of pre-set lighting styles (e.g., studio, outdoor, dramatic) and allow users to customize lighting parameters. The deliverable is a functional web service with a user-friendly interface and a subscription-based pricing model.

Create an interactive visualization using a simplified version of the IC-Light concept. Students will be provided with a set of images of a simple object (e.g., a sphere) under different single-source illuminations. They will then use sliders to linearly combine these images and compare the result with an image of the same object under a mixed illumination (sum of the individual sources). The deliverable is a presentation demonstrating the principle of linear light transport and a report explaining the observations.

Conduct a photographic experiment to verify the principle of consistent light transport. Capture a series of images of a static object under different controlled lighting conditions (single and mixed light sources). Then, using image editing software (without coding, only basic image manipulation), linearly combine the single-source images and compare the results (visually and quantitatively, e.g., using histograms) with the mixed-source images. The deliverable is a report documenting the experimental setup, results, and analysis, including a discussion of any discrepancies and their potential causes.

Extend the IC-Light framework to predict material BRDFs from a set of images captured under different lighting conditions. Instead of generating images, the model would learn to decompose the observed appearances into intrinsic material properties (BRDF) and illumination. The deliverable is a modified IC-Light model capable of BRDF prediction, a validation dataset, and a research paper reporting the findings.

Investigate the limitations of the linear light transport assumption in IC-Light and develop an extension to handle non-linear effects such as strong specular highlights, interreflections, and subsurface scattering. This could involve incorporating a non-linear component into the model architecture or developing a hybrid approach that combines IC-Light with a physics-based rendering engine. The deliverable is a modified IC-Light model capable of handling non-linear effects, a comprehensive evaluation dataset, and a research paper presenting the methodology and results.

Implement a simplified version of the IC-Light model and apply it to the task of relighting faces in a specific dataset (e.g., CelebA). Analyze the model's performance on different subsets of the data (e.g., varying skin tones, genders, ages) and identify any biases or limitations. The deliverable is a working implementation of the model, a report analyzing the results, and a presentation summarizing the findings.

Develop a simplified, user-friendly software application (e.g., web or mobile app) based on TANGO's core technology, allowing users to generate co-speech gesture videos of virtual characters from text and audio input. The deliverable is a functional application with a basic set of customizable avatar options and animation styles, along with a market analysis report on target user demographics.

Create a visualizer that demonstrates the relationship between audio features (pitch, volume) and simple 2D body movements (e.g., arm raising, head nodding). Use a pre-processed dataset of audio and corresponding simplified motion data (extracted from TANGO's data, but simplified to 2D). The deliverable is a program (e.g., in Python using libraries like Pygame) that displays the 2D motion synchronized with audio playback, along with a written report explaining the relationship observed.

Conduct a detailed analysis of the potential societal and ethical implications of widespread use of co-speech gesture video generation technology (like TANGO). This includes issues such as deepfakes, misinformation, job displacement, and accessibility. The deliverable is a comprehensive report outlining potential risks, benefits, and policy recommendations for responsible development and deployment of this technology.

Adapt TANGO's gesture generation model to control a physical robotic arm. Instead of generating video frames, the output of the adapted model will be control signals for the robot's actuators. The goal is to generate natural-looking arm gestures for the robot based on speech input. The deliverable is a working prototype demonstrating speech-driven gesture control of the robotic arm, along with a research paper evaluating the performance and limitations of the system, comparing it to existing methods.

Develop a generative model for co-speech gestures that does *not* rely on retrieval from a motion graph. This could involve exploring alternative architectures like Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), trained on a large dataset of co-speech gestures. The model should learn the underlying distribution of natural gestures and be able to generate novel gesture sequences conditioned on speech audio. The deliverable is a novel generative model architecture and implementation, with a comprehensive evaluation comparing its performance (using metrics like FVD, FGD, BC, and Diversity presented in the paper, as well as user studies) to TANGO and other existing methods.

Experiment with different architectural variations and loss functions for the AuMoCLIP component of TANGO. Investigate the impact of these changes on the retrieval accuracy and quality of generated gestures. For example, try different transformer architectures, contrastive loss variations, or adding additional modalities (e.g., text transcripts). The deliverable is a modified AuMoCLIP implementation, a detailed report on the experiments conducted, and a quantitative evaluation of the performance of different variations using metrics from the original paper.

Develop a real-time wildlife monitoring software prototype using SAM 2. The software will allow users to upload drone footage of wildlife areas and automatically identify, track, and count specific animal species. The software will feature a user-friendly interface for specifying target species (using prompts, similar to SAM 2), setting alert zones, and generating reports on animal populations and movements. Deliverables: a functional software prototype, user documentation, and a market analysis report.

Create a comparative study of visual complexity in different image datasets using the concepts presented in SAM 2. Students will manually segment a set of images (e.g., 100 images of everyday objects, 100 images of natural landscapes, and 100 images of abstract art) using simple drawing tools, focusing on identifying and outlining distinct objects. They will then compare and categorize the complexity of the segmentations based on the number of segments, their shapes, and their relationships. Deliverables: A report comparing the image datasets, a presentation of findings, and a poster illustrating different levels of visual complexity.

Define and evaluate the performance of SAM 2 for a novel robotic manipulation task. The task will involve identifying and grasping specific objects in a cluttered environment. The project will involve defining a series of increasingly complex scenarios (varying object types, clutter levels, lighting conditions), specifying prompts for SAM 2 to identify target objects, and qualitatively evaluating the model's success rate and identifying common failure modes. Deliverables: A detailed task definition document, a set of annotated test scenarios, a report evaluating SAM 2's performance, and a presentation of findings and recommendations for improvement.

Adapt and evaluate SAM 2 for segmentation of specific anatomical structures in 3D medical imaging data (e.g., MRI or CT scans). This will involve developing a method to convert 3D volumetric data into a format suitable for SAM 2's input, modifying the prompting mechanism to handle 3D spatial information, and evaluating the model's performance on a publicly available medical imaging dataset. Deliverables: Modified SAM 2 code, a detailed methodology document, a quantitative performance evaluation on a benchmark dataset, and a conference paper submission.

Develop a recurrent memory module for SAM 2 to improve its performance on videos with complex temporal dynamics (e.g., rapid object motion, occlusions, and appearance changes). This module will integrate with SAM 2's existing architecture and will be trained to explicitly model temporal dependencies between frames. The project will involve designing the module, implementing it in code, training it on a modified version of the SA-V dataset, and evaluating its performance against the original SAM 2 and other state-of-the-art video segmentation models. Deliverables: A novel recurrent memory module integrated with SAM 2, a detailed technical report, a quantitative performance evaluation, and a journal paper submission.

Extend the SA-V dataset by collecting and annotating a new set of videos focusing on a specific challenging scenario (e.g., scenes with significant camouflage, videos with very small objects, or videos with rapid camera motion). The project will involve defining a detailed annotation protocol, collecting at least 50 new videos, annotating them using the SAM 2 interactive annotation tool, and verifying the quality of the annotations. Deliverables: A new annotated video dataset, a detailed annotation protocol document, a report analyzing the challenges of the chosen scenario, and a presentation summarizing the project.

Develop a 'Schema Linking Assistant' software as a service (SaaS) product. This tool will leverage visual interfaces and natural language processing to help users map natural language questions to complex, large-scale database schemas. It will include a dialect-specific translation engine pre-trained on various enterprise database systems (BigQuery, Snowflake, SQLite, etc.). The deliverable is a working prototype with a user-friendly interface and integration with at least three database systems (BigQuery, Snowflake, and SQLite). The product should include unit tests, schema examples and comprehensive documentation.

Create a visual tool that helps users understand and construct simple SQL queries from natural language questions. Focus on a small, curated subset (10-20 examples) of the Spider 2.0-lite dataset using only SQLite databases, and limit queries to SELECT statements with simple WHERE clauses. The deliverable is a functioning program (e.g., using a visual programming language like Scratch or a simple web interface) that takes a natural language question, shows a visual representation of the database schema, and allows users to construct the corresponding SQL query by dragging and dropping elements.

Conduct a case study comparing and contrasting how different database systems (e.g., BigQuery, Snowflake, SQLite) represent the same underlying data. Select a small subset of related tables from Spider 2.0 and document how they are structured in each system. Analyze how the differences in schema and dialect would impact the process of writing SQL queries for the same informational need. The deliverable is a detailed report documenting the schemas, comparing the query formulation process for each system, and highlighting the challenges for a non-technical user.

Design and evaluate an alternative interactive interface for the Spider 2.0 agentic setting. Focus on improving the usability and transparency of the interaction between the LLM and the user, specifically addressing the challenges of navigating the codebase and understanding the database state. The deliverable includes a prototype interface, a user study comparing the new interface to the original setup (using a think-aloud protocol), and a detailed analysis of the usability improvements and remaining challenges.

Develop a novel program synthesis approach that combines neuro-symbolic reasoning with reinforcement learning to improve performance on the Spider 2.0 benchmark. Specifically, design a system that learns to decompose complex text-to-SQL tasks into a sequence of simpler sub-tasks, leveraging a symbolic representation of the database schema and SQL syntax. The reinforcement learning component should be used to optimize the exploration of the search space and handle the long-horizon planning required for multi-step workflows. The deliverable is a working system, a detailed technical report describing the methodology, and a comprehensive evaluation comparing the performance to the baselines reported in the Spider 2.0 paper, plus state-of-the art models.

Implement and evaluate a text-to-SQL parser specifically designed for one of the less common SQL dialects used in Spider 2.0 (e.g., ClickHouse or DuckDB). Start by creating a simplified grammar for the dialect, then develop a parser using a tool like ANTLR. Evaluate the parser's performance on a subset of the Spider 2.0-lite dataset relevant to that dialect, comparing it to the performance of a generic text-to-SQL model. The deliverable is a working parser, a report documenting the implementation and evaluation, and a set of test cases demonstrating the parser's capabilities and limitations.

Develop a specialized chatbot service based on OLMOE for a niche market, such as technical support for a specific software or a highly specialized educational tool. Create a web interface and API access. Conduct a market analysis report proving cost-effectiveness and superior performance to non-MoE.

Using the publicly available OLMOE model and preprocessed training data, create visualizations (e.g., heatmaps or bar graphs) showing the activation patterns of different experts for various input text categories (e.g., scientific papers, news articles, code). Prepare a presentation explaining the concept of model specialization.

Analyze a subset of the OLMOE pretraining dataset (e.g., a specific domain like arXiv papers or a specific time period) to assess the data quality. Create a report that identifies any potential biases, inaccuracies, or inconsistencies. Suggest concrete steps for improving the dataset based on criteria defined in existing data quality research.

Develop new metrics for quantifying domain and vocabulary specialization in MoE language models, drawing on techniques from computational linguistics or information theory. Apply these metrics to OLMOE and compare its specialization patterns with those of dense models, generating a comparative analysis report and proposing an interpretation using related theories.

Develop and implement a novel routing algorithm for Mixture-of-Experts language models that aims to improve expert specialization and reduce training time compared to the dropless token choice routing used in OLMOE. Evaluate the new algorithm on the OLMOE architecture using the provided training data and benchmark tasks, producing a conference-style paper with empirical results.

Replicate a subset of the pretraining experiments in the OLMOE paper, focusing on comparing token choice and expert choice routing. Document the replication process, analyze the results, and compare them with the findings reported in the paper, compiling a detailed technical report.

Develop a service that specializes in creating fine-tuning datasets and offers consulting for businesses to improve LLM code generation performance for niche libraries and complex, domain-specific instruction sets. The deliverable would be a platform offering curated datasets and expert consultation, with pricing based on dataset size/complexity and consulting hours. Performance improvement on BigCodeBench and client-specific benchmarks would be key metrics.

Manually create a dataset of 100 Python programming tasks, each with a natural language instruction, a corresponding code solution, and 3-5 test cases, focused on a single, commonly used Python library (e.g., `datetime`, `os`, or `math`). Instructions should involve multiple steps or conditional logic. The deliverable is a well-documented dataset in a structured format (JSON) suitable for augmenting BigCodeBench.

Select 50 tasks from the BigCodeBench-Complete dataset and create five natural language instruction variations for each, focusing on different phrasing, levels of detail, and implicit vs. explicit instructions. The deliverable is a dataset of 250 new instructions paired with the original task IDs and an analysis report categorizing the types of variations introduced.

Develop a set of novel metrics for quantifying the complexity and ambiguity of natural language instructions in code generation tasks, drawing on principles from computational linguistics or cognitive science. Evaluate these metrics on the BigCodeBench-Instruct dataset and correlate them with the performance of various LLMs. The deliverable is a peer-reviewed publication outlining the new metrics and their correlation with LLM performance.

Develop a semi-automated system to update BigCodeBench with new function calls and test cases reflecting changes in a specific set of rapidly evolving Python libraries (e.g., `tensorflow`, `pytorch`). This system should leverage API documentation, release notes, and code repositories to identify changes, generate new task descriptions, and create corresponding test cases. The deliverable will be a prototype system, a technical report describing the methodology, and an updated portion of BigCodeBench reflecting the latest library versions.

Extend BigCodeBench by adding 20 new programming tasks covering two Python libraries not currently included, focusing on libraries relevant to a specific application domain (e.g., bioinformatics or data visualization). This includes creating task descriptions, code solutions, and a comprehensive suite of test cases following the BigCodeBench format. The deliverable is a well-documented extension to the benchmark, including a report on the chosen libraries and the rationale for task selection.

Develop a platform for creating personalized, audio-driven avatars for video conferencing and virtual customer service applications. Users will be able to upload a photo and audio recording, and the system will generate a realistic avatar that mimics the user's speech and facial expressions. Deliverables include a functional web application, API documentation, and a market analysis report.

Investigate the correlation between specific audio features (e.g., pitch, volume) and corresponding facial movements (e.g., eyebrow raises, mouth opening) using a simplified dataset of audio recordings and facial landmark data. The deliverable is a report analyzing the correlations and visualizing the relationships between audio features and facial expressions.

Conduct a comparative analysis of existing datasets used for training audio-driven portrait video generation models. Identify potential biases (e.g., gender, ethnicity, age) and propose strategies for creating a more balanced and representative dataset. Deliverables include a report summarizing the analysis and recommendations for dataset improvement.

Incorporate a biomechanical model of facial muscles into the Loopy framework to constrain the generated motion and improve its anatomical accuracy. Evaluate the impact of these constraints on the perceived naturalness of the generated video. Deliverables include a modified version of Loopy with the integrated biomechanical model, and a research paper reporting the results.

Develop an adversarial training framework for Loopy to improve its robustness to variations in input audio and image quality, including different lighting conditions, head poses, and audio noise. The framework should include a discriminator network that specifically targets artifacts caused by challenging input conditions. Deliverables include a modified Loopy model with adversarial training, and a research paper reporting the results and comparing performance to the original model.

Experiment with different neural network architectures for the audio-to-latents module in Loopy, specifically focusing on improving the representation of subtle facial expressions. Compare the performance of different architectures (e.g., transformers, CNNs) using quantitative metrics and qualitative visual analysis. Deliverables include a report summarizing the experiments, modified code, and an analysis of the results.

Develop a commercial 'Hallucination Detection' API and user interface that integrates with popular LLMs. This API will utilize the methodology of identifying and analyzing uncertainty directions (as described in Section 7) to provide a 'confidence score' for LLM outputs. Deliverables include a working API, a user-friendly dashboard for visualizing confidence scores, and documentation for developers. The target market is businesses using LLMs for content generation, customer service, and data analysis.

Create a visual essay and interactive presentation exploring LLM hallucinations. Students will use a simplified, pre-compiled dataset of LLM responses (provided, based on the paper's dataset creation process, but pre-computed to eliminate coding) categorized by entity type (movie, city, song, player). They will manually analyze the responses, identifying instances of correct answers, refusals, and hallucinations. They will create visualizations (graphs, charts) to illustrate the frequency of each response type for known vs. unknown entities. The deliverables include a written report, visual aids, and a presentation suitable for a science fair.

Conduct a comparative analysis of refusal behavior across different LLMs (e.g., Gemma 2 2B, Gemma 2 9B, Llama 3.1 8B). Using a standardized set of prompts (based on the paper's dataset, but pre-run), analyze the responses and categorize them into: correct answers, refusals (with different justifications), and hallucinations. Develop a framework for qualitatively evaluating the 'quality' of refusals (e.g., clarity, helpfulness, consistency). The deliverables include a report summarizing the findings, the qualitative evaluation framework, and policy recommendations for improving LLM refusal mechanisms.

Design and conduct a series of experiments to compare the entity recognition patterns observed in LLMs (using the paper's SAE methodology) with human cognitive processes. This could involve: (1) creating analogous tasks for human participants involving entity recognition and recall under varying conditions (e.g., time pressure, uncertainty); (2) analyzing reaction times and error patterns; and (3) developing a computational model (informed by cognitive science principles) to explain both the human and LLM data. Deliverables include a peer-reviewed publication, presentation at a relevant conference (e.g., Cognitive Science Society), and potentially a publicly available dataset.

Investigate the use of nonlinear dimensionality reduction techniques (e.g., t-SNE, UMAP) in conjunction with SAEs to better visualize and understand the structure of entity representations in LLMs. Develop a novel method for 'fine-grained steering' that allows for manipulating specific aspects of entity knowledge (e.g., selectively altering an attribute while preserving others). Evaluate the effectiveness of this method using a more comprehensive set of attributes and entity types than the original paper. Deliverables include a peer-reviewed publication, open-source code implementing the new methods, and a demonstration of fine-grained steering capabilities.

Replicate and extend the 'steering' experiments from Section 5 of the paper. Using pre-trained SAEs (e.g., from Gemma Scope) and a provided set of prompts, reproduce the findings on refusal behavior modulation. Then, extend the experiment by: (1) testing different steering coefficients; (2) exploring the effects of steering with combinations of latents; and (3) analyzing the generated text for qualitative changes in style and content. Deliverables include a report documenting the replication and extension, code used for the experiments, and a presentation of the findings.

Develop a cloud-based service for accelerated molecular design using NF-BO. Target pharmaceutical companies and materials science research groups. The service will take SMILES strings or other molecular representations as input and optimize them based on user-defined objective functions (e.g., binding affinity, solubility). Deliverables include a web interface, API access, and reports summarizing optimization results. Develop a pricing model based on computational resources used and the complexity of the optimization task.

Create an interactive visualization of the value discrepancy problem described in the paper. Use a pre-trained Variational Autoencoder (VAE) on a simple dataset (e.g., MNIST handwritten digits) and show how reconstruction errors lead to different outputs even for similar latent representations. Implement this using a user-friendly library like p5.js or Processing. The deliverable is an interactive web page demonstrating the problem.

Analyze the performance of different optimization strategies in a non-coding context, such as resource allocation or project scheduling. Compare a 'naive' approach with an approach that utilizes a more accurate representation of the problem's constraints and dependencies. Document the differences in outcomes and explain how this relates to the value discrepancy problem discussed in the paper. The deliverable is a detailed report comparing the approaches and linking the findings to the core concept of NF-BO.

Adapt the SeqFlow model to perform image super-resolution. Train the model on a dataset of low-resolution and high-resolution image pairs. Instead of sequence tokens, use image patches as input. Evaluate the performance of the adapted model against existing super-resolution techniques (e.g., bicubic interpolation, SRGANs). The deliverables include the adapted SeqFlow model code, a report comparing its performance with existing methods, and a presentation summarizing the findings.

Extend NF-BO to support conditional molecular generation. Develop a method to incorporate desired properties (e.g., specific functional groups, target binding affinity) as conditions into the SeqFlow model. Evaluate the model's ability to generate molecules that satisfy these conditions, comparing its performance to existing conditional generation methods. Deliverables include the extended NF-BO code, a comprehensive evaluation report, and a publication-ready manuscript.

Implement the Token-level Adaptive Candidate Sampling (TACS) strategy and evaluate its impact on the performance of a simplified version of NF-BO. Use a pre-trained SeqFlow model and a standard Bayesian Optimization library (e.g., BoTorch). Compare the convergence speed and optimization results with and without TACS, using different temperature parameters. The deliverables include the TACS implementation, a report comparing its performance with standard sampling, and a presentation of the findings.

Develop a web service that allows users to generate variations of product images using the sCM (TrigFlow-based) approach. The service will take an input image of a product and generate multiple variations (different angles, backgrounds, lighting conditions) using a pre-trained sCM model. Deliverables include a functional web application, a user interface, and a performance evaluation report comparing generation speed and quality with existing solutions. The service should allow small batch generation for free and larger batch generation for a fee.

Using a pre-trained sCM model (simplified interface to be provided), explore the effects of different time steps and guidance scales on the generated images. Specifically, investigate how varying these parameters influences image clarity, diversity, and adherence to a given text prompt (e.g., 'a red apple'). Document the observations with example images and create a presentation summarizing the findings, including visual comparisons and a qualitative explanation of the observed trends.

Perform a comparative analysis of the training stability and sample quality between discrete-time CMs and continuous-time sCMs (TrigFlow) using pre-generated training logs and sample images. Evaluate the impact of different hyperparameters (provided) on each model type. Prepare a report summarizing the findings, focusing on quantitative metrics like FID scores and qualitative assessments of sample diversity and visual artifacts. Focus will be on evaluating claims in section 4 of the paper, not implementing anything.

Adapt the TrigFlow formulation for audio generation. Develop a continuous-time consistency model for generating high-fidelity audio samples (e.g., musical instrument sounds or speech). Investigate the applicability of adaptive weighting and tangent normalization techniques in the context of audio data. Compare the performance (sound quality, generation speed) of the proposed model with existing audio diffusion models, using objective metrics (e.g., spectral distortion) and subjective listening tests. The deliverable will be a conference-style paper reporting on adapting TrigFlow to audio generation.

Investigate the limitations of TrigFlow-based sCMs in high-guidance scenarios and propose a novel method to improve their performance. Explore incorporating techniques from classifier-free guidance or other advanced guidance methods into the TrigFlow framework. Develop a theoretical analysis of the proposed method, focusing on its impact on the training objective and sampling process. Empirically evaluate the proposed method on challenging image generation tasks requiring strong guidance (e.g., text-to-image generation with complex prompts) and compare its performance with existing CMs and diffusion models. Prepare a publication for a top-tier machine learning conference.

Implement the TrigFlow diffusion process and the associated continuous-time CM parameterization in PyTorch. Train a small-scale sCM model on a simple dataset (e.g., MNIST or CIFAR-10) and reproduce the two-step image generation results presented in the paper. Conduct ablation studies to evaluate the impact of tangent normalization and adaptive weighting on training stability and sample quality, by measuring and comparing FID scores. Prepare a report documenting the implementation, experimental results, and analysis, including code, visualizations, and quantitative comparisons.

Develop a business plan and user interface mock-up for a 'Bioisostere Generator' service based on ShEPhERD. The service will take a SMILES string as input and generate a set of potential bioisosteres. The output will include: (1) SMILES strings of the generated molecules; (2) Predicted 3D similarity scores (shape, electrostatics, pharmacophores) to the input molecule; (3) Drug-likeness scores (e.g., QED, SA Score); (4) Renderings of the 3D structures. The business plan will explore market size, competition (e.g., existing bioisostere databases), pricing models, and potential customers (pharma, biotech, materials science companies). The user interface mockup will showcase a simple, intuitive design for inputting molecules and visualizing results.

Create a visual tool (using a platform like Google Sheets or a simple web app) to compare the 3D shapes of known bioisosteres. The project will use a pre-generated dataset of molecules (e.g., from ZINC or a simplified subset of ShEPhERD's training data) and their corresponding shapes (represented as simplified point clouds or pre-rendered images). Students will manually input parameters for simplified shape similarity calculations (e.g., comparing overall size/volume, presence of key features represented as colored spheres). The tool will display the molecules and their calculated shape similarity scores, allowing students to explore how different molecules can have similar shapes despite having different chemical structures. Deliverables: the tool, a poster presentation explaining bioisosterism and shape similarity, and a report analyzing a selected set of bioisosteres.

Conduct a critical analysis of the ShEPhERD model's limitations and assumptions, focusing on its applicability to drug discovery. The report should address the following: (1) The impact of using context-free environments (no protein target) on the relevance of generated molecules. (2) The reliance on accurate partial charge calculations (via semi-empirical DFT) and how inaccuracies might affect the electrostatic similarity scores. (3) The limitations of the pharmacophore definitions used and how they might miss important interactions. (4) The potential biases introduced by the training data. (5) Comparison of ShEPhERD to other methods for lead optimization (literature review). Deliverable: A 5-page report addressing these points, supported by literature and examples where possible.

Adapt the ShEPhERD model for designing polymers with specific interaction profiles for applications in materials science. This will involve: (1) Defining appropriate interaction profiles for polymers (e.g., hydrophobicity, surface charge, specific binding motifs). (2) Creating or adapting a dataset of polymers with corresponding interaction profiles (this may involve simulations or experimental data). (3) Modifying the ShEPhERD architecture (if necessary) to handle polymer structures (e.g., longer chains, repeating units). (4) Evaluating the generated polymers using relevant metrics for the chosen application (e.g., mechanical properties, conductivity, binding affinity). Deliverable: A conference paper or report detailing the adapted model, the dataset, and the evaluation results. This will require substantial coding adaptation of the existing model.

Extend the ShEPhERD model to incorporate protein-ligand interactions, enabling structure-based drug design. This involves: (1) Developing a method for representing the protein binding site in a way that is compatible with the diffusion model (e.g., point cloud representation, voxel representation). (2) Modifying the diffusion process to jointly denoise the ligand and the protein binding site, while explicitly modeling their interactions. (3) Creating a suitable training dataset of protein-ligand complexes with interaction profiles. (4) Defining new metrics for evaluating the generated ligands, considering both binding affinity and interaction similarity. (5) Comparing the performance of the extended model to existing structure-based drug design methods. Deliverable: A publishable research paper demonstrating the extended model and its performance.

Implement and evaluate different conditioning methods for the ShEPhERD model. This project will explore alternatives to inpainting for conditioning the generation process. This may involve: (1) Implementing classifier-free guidance (using a trained classifier to guide the diffusion process). (2) Investigating different weighting schemes for the shape, electrostatic, and pharmacophore components during conditioning. (3) Exploring the use of different loss functions during training. (4) Evaluating the generated molecules using the metrics defined in the original paper, as well as additional metrics relevant to drug discovery (e.g., docking scores). Deliverable: A report detailing the implemented methods, the evaluation results, and a comparison to the original inpainting approach. A well-documented codebase is also a deliverable.

Develop an improved financial forecasting system using an LRNN with extended eigenvalue range. Benchmark its performance against a standard LRNN (with [0,1] eigenvalue range) and other established forecasting models (e.g., ARIMA, Prophet) using historical stock market data. The system should include data preprocessing, model training, prediction, and a backtesting framework to evaluate profitability. Deliverable: A functioning forecasting system with documented performance metrics and a comparison report.

Create a simplified parity checker in Excel. Model a single-state LRNN using a single cell to represent the state. Use an input column of 0s and 1s. Implement two update rules: one where the state is multiplied by 1 (analogous to eigenvalue in [0,1]), and another where it's multiplied by -1 (analogous to eigenvalue in [-1,1]). Observe how the state changes with each input and whether it correctly tracks parity (even or odd number of 1s). Deliverable: An Excel spreadsheet with the implemented model and a short report comparing the behavior of the two update rules.

Analyze a corpus of text (e.g., news articles or short stories) and identify sentences where correct interpretation *requires* tracking the state of entities or relationships across multiple clauses. Create a dataset of these sentences, annotated with the relevant state information. Compare the difficulty of interpreting sentences where the key state information is expressed positively versus negatively (e.g., "The cat, *which was black*, sat on the mat" vs. "The cat, *which was not white*, sat on the mat"). Deliverable: Annotated dataset and a report analyzing the prevalence and complexity of state-tracking in natural language, with specific attention to the role of negative constructions.

Apply control theory principles (e.g., Lyapunov stability) to analyze the stability and robustness of LRNNs with different eigenvalue ranges. Develop a theoretical framework for characterizing the relationship between the eigenvalue spectrum of the state-transition matrix and the network's sensitivity to noise or perturbations in the input. Deliverable: A conference-style paper presenting the theoretical analysis and (optionally) simulation results.

Investigate the integration of attention mechanisms into LRNNs with extended eigenvalue ranges. Develop a novel attention mechanism that selectively weights hidden states based on both positive and *negative* relevance, inspired by the concept of negative eigenvalues. Evaluate the performance of this "signed attention" LRNN on challenging state-tracking tasks and compare it to standard attention mechanisms and baseline LRNNs. Deliverable: A conference-style paper reporting the new architecture, theoretical justification, and experimental results; open-source code implementation.

Replicate the parity and modular arithmetic experiments from the paper using a standard deep learning framework (e.g., PyTorch or TensorFlow). Extend the experiments to a new dataset, such as a dataset of simple arithmetic expressions (e.g., "2 + 3 - 1 = 4"). Compare the performance of LRNNs with [0,1] and [-1,1] eigenvalue ranges on this new dataset. Deliverable: Python code implementing the models and experiments, a report documenting the results and comparing them to the original paper, and an analysis of any observed differences.

Develop a prototype 'SAE Bias Detector' tool. This tool would ingest activations from a large open-source language model (e.g., Llama, Mistral) run on specific prompts designed to elicit biases. Using a pre-trained large SAE (potentially building on the paper's released models or training a new one), calculate the ablation sparsity metric (Section 4.5) for features activated by these prompts. The tool would flag features with high activation and sparse, significant negative downstream effects (indicating strong influence on potentially harmful outputs) for human review. Deliverables: A functional web-based prototype demonstrating the workflow on 2-3 types of bias, and a business plan outlining target customers (e.g., AI developers, auditing firms) and market opportunity.

Comparative Analysis of Language Model Features using an SAE Visualizer. Using the feature visualizer released with the paper for the GPT-2 small autoencoders, select 10 distinct features identified by the SAE (e.g., 5 related to syntax like 'end of quote', 5 related to semantics like 'positive sentiment'). For each feature, document its top activating examples provided by the visualizer. Analyze and compare the activation patterns: what text patterns consistently trigger each feature? How similar or different are the patterns for features within the same category (syntax vs. semantic)? Deliverable: A written report summarizing the findings for each feature, including screenshots from the visualizer, and a comparison of the observed patterns.

Qualitative Evaluation of High-Level Features in GPT-4 Activations. Using the paper's feature visualizer for the 16 million latent GPT-4 autoencoder, identify and select a thematic cluster of 15-20 features (e.g., features activating on code snippets, legal text, or specific emotional tones). For each selected feature, analyze its top activating examples and random activations. Attempt to formulate a concise natural language description (hypothesis) of the concept the feature represents. Assess the 'monosemanticity' qualitatively – do most activating examples fit the hypothesis? Document examples that seem to contradict the hypothesis. Deliverable: A curated list of the features, their hypothesized descriptions, supporting/contradicting examples, and a qualitative assessment of their interpretability.

Sparse Autoencoder Feature Discovery in Neural Time-Series Data. Adapt the k-sparse autoencoder methodology (TopK activation, transpose initialization, auxiliary dead latent loss) described in the paper to analyze high-dimensional time-series data from another field, such as multi-electrode neural recordings or eye-tracking data during a cognitive task (e.g., reading). Train SAEs on segments of this data. Analyze the discovered latent 'neural/behavioral features' for interpretability within the domain context. Compare the statistical properties (e.g., sparsity distribution, activation frequency, reconstruction quality) of these features with those reported for language model features in the paper. Deliverable: A research paper presenting the adapted methodology, analysis of the discovered features, and comparison with the original paper's findings on LMs.

Developing and Evaluating Adaptive-k Sparse Autoencoders. Design and implement a modification to the k-sparse autoencoder architecture where the number of active latents, k, is not fixed but dynamically determined per input token. Explore methods like: 1) Training an auxiliary small network that predicts an optimal k' for each token based on the input activation, using reconstruction loss and a penalty on k' in the objective. 2) Using reinforcement learning to learn a policy for selecting the number of latents. Train this 'Adaptive-k SAE' on language model activations (e.g., GPT-2 or larger) and compare its performance on the reconstruction-sparsity (L0 vs MSE) frontier and downstream evaluation metrics (probe loss, downstream loss, explainability) against the fixed-TopK SAEs from the paper. Deliverable: A research paper detailing the proposed adaptive-k method(s), experimental results, and analysis compared to the baseline fixed-k approach.

Comparative Analysis of TopK vs. L1 Regularization for Activation Shrinkage in SAEs. Implement a TopK sparse autoencoder based on the description in Section 2.3 of the paper, using released code for GPT-2 small as a base. Train this SAE on layer 8 activations of GPT-2 small. Separately, train a standard ReLU autoencoder using L1 regularization (Section 2.2) tuned to achieve a similar average L0 sparsity as the TopK model. Evaluate both models on reconstruction MSE and L0 sparsity. Crucially, implement the 'refinement' procedure described in Section 5.1 (optimizing activations for a fixed decoder) for both models. Compare the magnitude and bias of the refinement adjustments (as in Figure 8a) and the resulting improvement in MSE and downstream loss (Figures 8b, 8c) to empirically test the paper's claim that TopK prevents activation shrinkage compared to L1. Deliverable: Documented code for the TopK SAE and refinement procedure, and a report comparing the performance and refinement results of the TopK and L1 SAEs.

Develop a commercial software tool that uses the BirdSet dataset to provide real-time bird species identification and population density estimates for a specific geographic region (e.g., national parks). This tool should allow users to upload audio data and receive automated reports, including species presence, abundance, and potential threats. The deliverable is a functional software product with a user-friendly interface and clear reporting features, a business plan and go-to-market strategy.

Using a pre-trained model from the BirdSet project and a simplified interface (e.g., a Google Colab notebook or a GUI-based machine learning tool), classify bird vocalizations from a small, curated subset of the BirdSet evaluation data. Focus on 5-10 common bird species in a specific region and analyze the model's performance. The deliverable is a report analyzing the classification results, including a confusion matrix and error analysis.

Conduct a thorough quality control assessment of a subset of the BirdSet dataset (e.g., 1000 recordings). This involves listening to the recordings, verifying the provided labels, and identifying any inconsistencies or errors in annotations (e.g., mislabeled species, missing vocalizations, incorrect time stamps). The deliverable is a detailed report outlining the identified issues and suggesting improvements to the dataset's annotation guidelines.

Apply advanced signal processing techniques (e.g., wavelet transforms, time-frequency analysis, source separation) to the BirdSet dataset to extract novel features for bird sound classification. Compare the performance of standard machine learning models (e.g., SVM, Random Forest) using these new features against the baseline features used in the BirdSet paper. The deliverable is a peer-review quality publication demonstrating the effectiveness of the proposed signal processing techniques and a feature set library.

Develop a novel deep learning model architecture that addresses the challenges of label noise and covariate shift in the BirdSet dataset. This could involve incorporating techniques such as semi-supervised learning, transfer learning, or adversarial training. Evaluate the model's performance on different subsets of the BirdSet data, specifically focusing on scenarios with high label noise or significant covariate shift. The deliverable is a novel model architecture, and a publication at a top-tier machine learning or bioacoustics conference.

Implement and evaluate different data augmentation strategies (beyond those mentioned in the paper, such as pitch shifting, time stretching, adding different types of environmental noise) on a subset of the BirdSet training data. Train a standard convolutional neural network (CNN) model with and without these augmentations and compare their performance on the BirdSet evaluation data. The deliverable is a report analyzing the impact of different data augmentation strategies on model performance, including metrics such as precision, recall, and F1-score.

Develop a cloud-based service using Grendel for accelerated 3D model generation from user-provided image sets. Offer tiered pricing based on model complexity (number of Gaussians) and rendering resolution. Deliverables: a functional web service, pricing model, and a marketing plan targeting e-commerce, real estate, and gaming industries.

Create an interactive web application using a pre-trained Grendel model (simplified or limited in scale) that allows users to visualize how changing the number of Gaussians and viewing angle affects the rendering of a simple 3D scene. Deliverables: a working web application, a short report explaining the observed effects, and a presentation suitable for a science fair.

Conduct a parameter sensitivity analysis of the Grendel system using pre-generated training logs with varying batch sizes and GPU counts. Analyze the relationship between batch size, learning rate, training time, and rendering quality (PSNR). Deliverables: a report summarizing the findings, including visualizations of performance trends and recommendations for optimal parameter settings in different scenarios.

Adapt the Grendel framework to accelerate a specific CFD simulation (e.g., airflow over an airfoil) by replacing the Gaussian representation with a suitable CFD discretization method. Implement a mixed parallelism strategy and dynamic load balancing tailored to the CFD problem. Deliverables: a modified Grendel codebase, performance comparisons against a standard CFD solver, and a conference paper outlining the methodology and results.

Develop an extension to Grendel that allows for incremental incorporation of new viewpoints during training. Implement a mechanism to dynamically adjust the Gaussian distribution and optimize parameters based on newly acquired images, without retraining from scratch. Deliverables: an extended Grendel codebase, a comparison of the proposed approach against retraining from scratch, and a publication in a leading computer graphics or computer vision conference.

Implement and evaluate alternative dynamic load balancing algorithms within the Grendel framework. Compare the performance (training time and rendering quality) of these algorithms against Grendel's default iterative rebalancing strategy using a standard dataset. Deliverables: a modified Grendel codebase, a performance evaluation report with quantitative results, and a presentation suitable for a capstone project.

Develop a commercial service focused on curating and annotating parallel corpora for extremely low-resource languages. This service will utilize a combination of web scraping, linguistic fieldwork (potentially crowdsourced), and automated quality control to build a database of high-quality parallel sentences. Deliverables include a subscription-based API for accessing the data, tools for data annotation and quality assessment, and consulting services for clients needing customized datasets. The focus will be on identifying languages with the least existing digital data and greatest need for language technology resources, establishing a unique market niche (an endangered language translation data marketplace).

Create a small parallel corpus for a low-resource language (potentially a local dialect or minority language) by collecting and translating example sentences. Students will identify a target language, collect text data (e.g., from stories, conversations, or simple documents), translate these sentences into English, and then analyze the challenges and limitations encountered during this process. Deliverables include the collected parallel corpus, a report documenting the data collection methodology, and an analysis of the language's grammatical features compared to English, based solely on the collected examples (not a grammar book).

Conduct a comparative analysis of different types of linguistic resources (grammar books, dictionaries, typological databases) for a selected low-resource language. Evaluate the strengths and weaknesses of each resource type for different NLP tasks (e.g., machine translation, morphological analysis, information retrieval). The deliverable is a report outlining the findings, including recommendations for prioritizing resource development based on the specific needs of the language and potential NLP applications. This *does not* involve building any NLP systems, only evaluating linguistic resources with respect to NLP applications described in the original paper (and elsewhere).

Design and conduct a human experiment to investigate how learners acquire a novel grammatical feature of a constructed or low-resource language. Compare learning outcomes when participants are provided with (a) grammatical explanations, (b) parallel examples, and (c) a combination of both. The experiment should be designed to mirror the setup of the LLM experiments in the paper, allowing for a direct comparison between human and machine learning. Deliverables include the experimental design, collected data, statistical analysis, and a report comparing human learning patterns to the LLM findings in the paper. Explore parallels with few-shot learning in humans and LLMs, relating performance to the different input data types (explanation vs example).

Develop and evaluate novel prompting strategies for incorporating grammatical explanations from grammar books into LLMs for low-resource machine translation. Investigate methods such as: (1) transforming explanations into structured formats (e.g., rule-based representations); (2) generating synthetic examples from grammatical rules; (3) designing interactive prompts that allow the LLM to query specific aspects of the grammar; (4) creating prompts which require step-by-step translation which utilize rules from the grammar book. Evaluate these strategies on a low-resource MT task, comparing performance against baselines using only parallel data and simple grammar book concatenation (as in the original paper). The deliverables include a new set of prompting strategies, a quantitative evaluation of their performance on the MT task, and a qualitative error analysis to understand their strengths and weaknesses. Develop a new benchmark for prompting LLMs with grammatical information for machine translation, including metrics to assess how effectively the LLM is using the grammar in its output.

Implement a simple encoder-decoder translation model for a low-resource language pair using only parallel sentence data (no grammar book). Experiment with different data preprocessing techniques (e.g., tokenization, subword segmentation) and model hyperparameters to optimize translation performance. Evaluate the model using standard MT metrics (e.g., BLEU, CHRF++) and compare the results to the LLM-based approach described in the paper. The deliverables include the implemented translation model, a report documenting the experiments and results, and an analysis of the model's errors and limitations.

Develop a SaaS tool that analyzes pre-trained language models to detect and quantify the presence and impact of attention sink. The tool should provide metrics and visualizations indicating the degree to which a model relies on attention sink, along with recommendations for optimization or alternative model architectures (e.g., suggesting sigmoid attention). Deliverables include a functional web application, API documentation, and a pricing model.

Using a small, pre-trained language model (e.g., a small GPT-2 model from Hugging Face) and a simple text dataset (e.g., a collection of short stories), visualize the attention weights for different input sequences. Specifically, create heatmaps of the attention matrices to observe the presence of attention sink (high attention to the first token). Experiment with different input types (e.g., natural language, random tokens, repeated tokens) as described in the paper and document the observed changes in attention patterns. Deliverables include a presentation with the visualizations and a written report summarizing the findings and comparing them to the paper's results.

Conduct a comparative analysis of different commercial language model APIs (e.g., OpenAI, Cohere, AI21 Labs) based on their potential susceptibility to attention sink. This analysis should rely on publicly available information, white papers, and technical documentation. Develop a framework for evaluating the efficiency of these APIs based on factors identified in the paper as influencing attention sink (e.g., model architecture, training data). Produce a report comparing the different APIs and recommending best practices for choosing and using them to minimize the negative impacts of attention sink. Focus should be put on business impact, not technical implementations.

Develop an information-theoretic framework to analyze the attention sink phenomenon in language models. Specifically, explore how attention sink affects the mutual information between different parts of the input sequence and the output. Investigate whether the presence of attention sink corresponds to a reduction in the effective information processing capacity of the model. Derive theoretical bounds or metrics to quantify the information loss due to attention sink. The deliverables include a peer-reviewed publication.

Investigate the use of a 'dynamic attention mask' to mitigate attention sink in autoregressive language models. This dynamic mask would learn to modulate attention weights based on the input sequence, potentially reducing the over-reliance on the first token. Experiment with different architectures for the dynamic mask (e.g., a small neural network that takes the hidden states as input) and different training strategies (e.g., jointly training the mask with the main language model or using a reinforcement learning approach). Evaluate the proposed approach on benchmark datasets and compare its performance to models with standard attention and other attention modifications (e.g., sigmoid attention). The deliverable is a peer-reviewed publication.

Replicate the experiments from the paper on the emergence of attention sink during LM pre-training, using a smaller model (e.g., a smaller version of LLaMA or GPT-2) and a subset of the Pile dataset. Investigate the impact of different learning rates and weight decay ratios on the emergence of attention sink, as described in the paper. Extend the experiments by exploring the effect of different optimizers (e.g., AdamW vs. SGD). Document the results and compare them to the findings presented in the paper. Deliverables include a well-documented code repository and a technical report.

Develop a commercial software tool that leverages the MMQA framework and MTR method to provide multi-table question answering capabilities for a specific industry vertical (e.g., financial analysis, supply chain management). The software should include a user-friendly interface for querying multiple databases and visualizing the reasoning process. Deliverables: functional software prototype, user documentation, market analysis for the chosen vertical, business plan. Measure success by user adoption, query accuracy and customer satisfaction.

Create a simplified version of the MMQA dataset using publicly available datasets (e.g., government statistics, sports data) that involves joining at least three tables. Develop a set of multi-hop questions that require reasoning across these tables. Manually create a 'gold standard' answer key. Evaluate the answers of classmates using simple metrics like precision and recall, similar to the paper. Deliverables: curated dataset, question set, answer key, evaluation results, presentation of findings.

Using a spreadsheet program (like Excel or Google Sheets), design a scenario involving at least three related tables (e.g., customer data, order data, product data). Formulate multi-hop questions that require joining and filtering data across these tables. Manually perform the necessary steps in the spreadsheet to answer these questions, documenting each step. Compare the efficiency and accuracy of different approaches to answering the same question. Deliverables: spreadsheet with data and questions, documented steps for each query, comparison of different query strategies, presentation of findings.

Apply graph-based analysis techniques from network science to model the relationships between tables in the MMQA dataset (or a similar multi-table dataset). Develop a method to quantify the complexity of multi-table joins based on graph properties (e.g., centrality, connectivity). Investigate the correlation between graph complexity metrics and the performance of LLMs on corresponding questions. Deliverables: graph representation of the dataset, complexity metrics, correlation analysis, research paper, presentation at a conference.

Extend the MMQA benchmark by introducing controlled noise and uncertainty into the tables (e.g., missing values, conflicting information, probabilistic relationships). Develop new evaluation metrics that are robust to noise. Design and evaluate new or modified LLM architectures that can effectively handle uncertainty in multi-table reasoning. Deliverables: extended MMQA dataset, new evaluation metrics, implementation of modified LLM architectures, research paper targeting a top-tier conference/journal, comparison of new and old results.

Implement a simplified version of the Multi-Table Retrieval (MTR) method proposed in the paper using a programming language like Python. Use a subset of the MMQA dataset or a smaller, publicly available multi-table dataset. Compare the performance of your MTR implementation with a baseline retrieval method (e.g., BM25). Evaluate performance using precision and recall. Deliverables: Python code for MTR implementation, comparison with baseline, report documenting the methodology and results, presentation.

Develop a cloud-based service that offers optimized LoRA fine-tuning using LoRA-RITE. The service will allow users to upload their datasets and models, and automatically apply LoRA-RITE with optimized hyperparameters. The service will provide performance benchmarks comparing LoRA-RITE to Adam and other standard optimizers, showcasing the benefits in terms of accuracy and training time. Offer different pricing tiers based on model size and computational resources used.

Create an interactive visualization using a tool like Desmos or GeoGebra to demonstrate the concept of transformation invariance in LoRA. Represent LoRA factors A and B as 2D vectors, and show how different linear transformations (scaling, rotation) affect the product AB (which represents the weight update Z). The visualization should allow users to manipulate A and B, apply different transformations, and observe the effect on Z. Compare the behavior with a 'non-invariant' transformation to illustrate the difference.

Conduct a comparative case study of LoRA-RITE versus Adam for a specific fine-tuning task (e.g., sentiment analysis on a public dataset). Document the entire process: data preparation, model setup, hyperparameter tuning (using available tools, no coding needed), and evaluation. Focus on the *practical* differences: ease of use, training time, resource consumption, and final performance. Present the findings in a report, emphasizing the implications for practitioners.

Adapt the LoRA-RITE algorithm for optimizing low-rank matrix factorization in recommendation systems. Instead of LLMs, apply the method to a collaborative filtering problem, replacing the weight matrices with user and item embedding matrices. Develop a theoretical analysis of transformation invariance in this context, and empirically evaluate the performance against standard matrix factorization optimizers on a public recommendation dataset (e.g., MovieLens).

Develop a hybrid optimizer that combines LoRA-RITE with K-FAC (Kronecker-Factored Approximate Curvature). Explore different strategies for integrating K-FAC's curvature information with LoRA-RITE's transformation-invariant preconditioning. Provide a theoretical analysis of the combined method, addressing convergence and transformation invariance. Empirically evaluate the performance on a suite of LLM benchmarks, comparing it to both LoRA-RITE and K-FAC.

Implement the LoRA-RITE algorithm in PyTorch and reproduce the results on the GSM8K dataset, comparing its performance with Adam. Conduct a sensitivity analysis of LoRA-RITE's hyperparameters (learning rate, momentum, second moment decay) and document their impact on training speed and final accuracy. Create a detailed report including the implementation, experimental results, and analysis of hyperparameter sensitivity.

Develop a 'CoT Reasoning Enhancer' module that can be integrated with existing large language models (LLMs) via an API. This module will specialize in a commercially relevant reasoning task (e.g., identifying logical inconsistencies in customer support requests or generating specific types of database queries from natural language). The deliverable is a functional API, a demonstration of its integration with a popular LLM, and a benchmark report showing improved performance on the chosen task compared to the base LLM without the enhancer. Evaluate cost-effectiveness by comparing performance gains against the computational cost of running the enhancer.

Create a physical game or interactive spreadsheet that demonstrates the k-parity problem with k=4 and d=8. Students will manually simulate the CoT process, creating intermediate steps to solve the parity of subsets of bits. The deliverable is a playable game or a working spreadsheet, along with a short report explaining how it relates to the concept of Chain-of-Thought reasoning.

Identify a complex, multi-step decision-making process within a specific professional domain (e.g., supply chain management, medical diagnosis, legal reasoning). Analyze how this process could be broken down into smaller, more manageable sub-tasks, analogous to the Chain-of-Thought approach. Develop a detailed workflow diagram or flowchart illustrating this decomposition, and write a report explaining how this approach could potentially improve efficiency, accuracy, or transparency compared to the current process. Include specific examples of sub-tasks and how they contribute to the final decision.

Design and conduct a cognitive science experiment to investigate whether human problem-solving on a task analogous to the parity problem (e.g., a complex rule-based categorization task) shows evidence of a Chain-of-Thought-like process. Analyze reaction times, error rates, and potentially eye-tracking or neuroimaging data to identify intermediate steps and decision points. The deliverable is a research paper suitable for submission to a cognitive science conference or journal, presenting the experimental design, results, and their interpretation in relation to the transformer CoT model.

Investigate the applicability of the Chain-of-Thought approach and the associated efficiency results to a broader class of computational problems beyond parity. Develop a theoretical framework that characterizes the types of problems for which CoT provably improves transformer performance. The deliverable is a theoretical paper suitable for submission to a machine learning conference (e.g., ICLR, NeurIPS), presenting the generalized framework, proofs, and potentially supporting simulations with alternative transformer architectures or problem types, e.g. exploring the capabilities of 2-layer transformers.

Implement a one-layer transformer model in Python using a deep learning library (e.g., PyTorch, TensorFlow) and train it to solve the k-parity problem with k=4 and d=16, using both end-to-end training and the Chain-of-Thought approach with teacher forcing. Compare the training time, convergence rate, and final accuracy of the two methods. The deliverable is a working Python implementation, a report documenting the experimental setup, results (including plots of training curves), and analysis of the differences between the two training approaches. Experiment with including self-consistency.