[SPARK-51216][BUILD] Remove the useless bigtop-dist profile and the related outdated files

### What changes were proposed in this pull request?
This PR removes the unused `bigtop-dist` configuration file, related configuration files, and outdated `README` file from the `assembly` module, as Spark now uses `dev/make-distribution.sh` to build the distribution instead of this profile.

### Why are the changes needed?
Remove the useless `bigtop-dist` profile and the related outdated files

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass Github Actions
- Manual check: the modifications will not affect the distribution build.

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#49952 from LuciferYang/remove-profile-bigtop-dist.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>

Author: LuciferYang

assembly/README

@@ -277,34 +277,6 @@
         </dependency>
       </dependencies>
     </profile>
-    <profile>
-      <id>bigtop-dist</id>
-      <!-- This profile uses the assembly plugin to create a special "dist" package for BigTop
-           that contains Spark but not the Hadoop JARs it depends on. -->
-      <build>
-        <plugins>
-          <plugin>
-            <groupId>org.apache.maven.plugins</groupId>
-            <artifactId>maven-assembly-plugin</artifactId>
-            <version>3.1.0</version>
-            <executions>
-              <execution>
-                <id>dist</id>
-                <phase>package</phase>
-                <goals>
-                  <goal>single</goal>
-                </goals>
-                <configuration>
-                  <descriptors>
-                    <descriptor>src/main/assembly/assembly.xml</descriptor>
-                  </descriptors>
-                </configuration>
-              </execution>
-            </executions>
-          </plugin>
-        </plugins>
-      </build>
-    </profile>
 
     <!-- Profiles that disable inclusion of certain dependencies. -->
     <profile>